Use of Hough Transform and Homography for the Creation of Image Corpora for Smart Agriculture

Use of Hough Transform and Homography for the
Creation of Image Corpora for Smart Agriculture
Jose A. Brenes1
, Javier Ferrández-Pastor2
, José M. Cámara-Zapata3
, and
Gabriela Marín-Raventós1
1
CITIC/ECCI/PPCI, University of Costa Rica, San José 11501, Costa Rica
joseantonio.brenes@ucr.ac.cr, gabriela.marin@ucr.ac.cr
2
I2RC, University of Alicante, Alicante 03690, Spain
3
CIAGRO-UMH, University Miguel Hernández, Alicante 03202, Spain
Abstract. In the context of smart agriculture, developing deep learning models demands large and high-
quality datasets for training. However, the current lack of such datasets for specific crops poses a significant
challenge to the progress of this field. This research proposes an automated method to facilitate the
creation of training datasets through automated image capture and pre-processing. The method’s efficacy
is demonstrated through two study cases conducted in a Cannabis Sativa cultivation setting. By leveraging
automated processes, the proposed approach enables to create large-volume and high-quality datasets,
significantly reducing human effort. The results indicate that the proposed method not only simplifies
dataset creation but also allows researchers to concentrate on other critical tasks, such as refining image
labeling and advancing artificial intelligence model creation. This work contributes towards efficient and
accurate deep learning applications in smart agriculture.
Keywords: Dataset Creation, Image Capturing and Pre-processing, Homography, Hough Transform,
Smart Farming.
1 Introduction
We are experimenting with an Artificial Intelligence (AI) boom where models have shown
an unprecedented potential to address distinct challenges and improve everyday life in
multiple fields. AI applications flourish daily in areas like Education, Entertainment, Cus-
tomer Services, Industry, and Agriculture. However, the success and performance of these
AI models are highly dependent on the quality and quantity of the data available to train
them [1].
There is a shortage of high-quality datasets for training AI applications. Although it is
possible to access a few public datasets in distinct areas of application [2, ?], there is still
a lack in other areas. In many domains, the existing datasets may be limited, outdated,
or not cover all possible variations of the problem to be solved. Therefore, creating new
datasets becomes essential to cover a greater range of real-world scenarios and situations.
Furthermore, in some emerging and specialized areas, such as medicine and robotics,
specific and properly annotated data is scarce or expensive to obtain. In this context,
generating new datasets can open new opportunities to train models that address specific
challenges in these fields and, ultimately, improve decision-making.
The scarcity of public datasets in smart farming is not asunder of the trend. It remains
a crucial bottleneck for creating computer vision and machine learning applications in this
domain [2]. Although it is possible to use technologies like drones to obtain large volumes
of data from the field, it is not always possible. Very restricted crops make it difficult to get
International Journal on Cybernetics & Informatics (IJCI) Vol.12, No.6, December 2023
Dhinaharan Nagamalai (Eds): NLAII, VISCOM, CCSITA, AIDD, BCYIoT -2023
pp. 09-19, 2023. IJCI – 2023 DOI:10.5121/ijci.2023.120602

data from cultivations. Still, there are also cultivation methods that make it challenging
to gather imaging data in fields, like cultivation under greenhouses.
Since data scarcity is a major challenge when training AI and deep learning (DL)
models [4], creating new training datasets becomes a motivation to continue advancing in
the research and application of AI. Another key reason to create new datasets lies in ethics
and fairness. Deep learning models can be affected by inherent biases in the training data.
By designing more diverse and balanced datasets, bias propagation can be reduced and
promote greater effectiveness and fairness in AI applications.
In this paper, we propose an automated method to capture image datasets for smart
farming AI applications. We propose a method to capture and pre-process the images,
taking into consideration adjustable time slots. This enables us to get a large volume of
clean and different images. This way, we avoid using augmentation techniques to train AI
models.
We used the proposed method in two study cases: first, we created a germination dataset
containing images of plants growing in seedbeds. In a second study case, we created an
image dataset of plants in different growing stages. Currently, we are working on separate
projects using the created datasets to train deep learning models to classify new samples.
The structure of this paper is as follows: first, we present related works about dataset
creation in distinct domains and some reviews related to publicly available datasets in the
area of smart farming applications. Next, we present our proposed method to automate
the capture and pre-processing of images for smart farming AI applications. Following, we
present the results obtained in two study cases related to creating image datasets. Finally,
we present some conclusions about the conducted research and future work.
2 Related work
Some studies analyzing the data scarcity of training datasets for AI applications have
been conducted. We consider it important to highlight some of them. First, the study
presented in [1] analyzes the problem derived from using the currently available datasets
to train models leaving aside the data quality and avoiding creating new ones. The authors
demonstrated this by using probabilistic models to estimate the quantity of data available
in the coming years. This way, the authors conclude that by around 2030 and 2060, AI
progress will probably slow down due to data scarcity.
In a recent study [4], authors present the problem related to data scarcity when training
AI models. In this study, authors also propose some solutions to data scarcity: (1) Transfer
Learning to take advantage of pre-trained models and re-applied them to new models, (2)
Self-supervised Learning to use large amounts of unlabelled data and small amounts of an-
notated data, (3) Generative Adversarial Networks as a tool to generate new data samples
with similar features as the real input data, (4) Model architecture, as a way to reduce the
complexity of the model to require fewer amounts of training data, (5) Physics-informed
neural networks, as a deep learning technique that can deal with insufficient training data,
and (6) Deep synthetic minority oversampling technique, to generate synthetic images to
address the problem of imbalanced data. Nevertheless, the authors do not mention, as
a solution, the possibility of creating new custom-made datasets, which we consider the
better option if possible.
From the study mentioned above, we also rescue an essential statement. The authors
pointed out a list of tips for reporting datasets. We consider this crucial because if the
details of the used dataset are clearly stated, it makes it possible for the reader of the
study to know about its origin, availability, and other technical details.
10

Regarding the availability of datasets, in [2], the authors described a list of public
image datasets in the precision agriculture domain. In the study, the main characteristics
of 34 datasets are analyzed: 15 datasets on weed control, ten datasets on fruit detection,
and nine datasets on miscellaneous applications. Besides, the authors concluded that the
presented survey can be valuable for researchers when they have to select an image dataset
to create new models. Furthermore, the study reveals the areas where the creation of new
datasets to support smart farming is needed.
No studies were found about techniques or systematic methods for creating new datasets
for smart farming. Nevertheless, some studies in other domains do describe them. In [5],
authors present a methodology based on processing screenshots to create an image dataset
for training Google’s t-rex game. Meanwhile, authors in [6] propose a framework devel-
oped to create human pose datasets, including camera calibration, refined human pose
estimation, and manual annotations.
The relevance of these studies relies on the systematization of the capturing method
and pre-processing of samples. We consider both requirements for our proposal. In the next
section, we present our proposed method to automate the capture and pre-processing of
images for smart farming AI applications.
3 Automating the capture and pre-processing of images in smart
farming
Most image datasets available for smart farming analyzed in [2] have problems like the
dataset size and the samples’ size. The first is related to the number of samples included
in the dataset, while the second is related to the size in pixels of each sample. We deal
with both problems in our proposal by accomplishing two requirements: (1) automating
the capture of the images and (2) processing the images to resize them as required.
To address the first requirement, we prepared a setup for image acquisition. It uses a
low-cost camera (Arducam 64MP Auto-focus camera, which can take images of 9152x6944
pixels) and a Raspberry Pi to control it. We place and fix the camera and the Raspberry
Pi inside a greenhouse. Then, we created a Python script to manage the distinct camera
options (focus, zoom, size, among others) and to capture the images automatically. This
way, we can adjust the camera sensor according to the climate conditions and schedule the
capture of the images. After some tests, we discovered that under greenhouse conditions,
the camera sensor takes a maximum of twelve seconds to take each picture. Thus, we could
easily take at least one image every fifteen seconds.
Fig. 1 shows the camera setup inside the greenhouse. It is crucial to note that our
proposed method requires placing circle reference markers containing the study area. In
Fig. 1, reference markers are detailed as pink circles delimiting various plant pots. These
reference markers will be used to pre-process the captured images as it is described below.
As seen in Fig. 1, we can rotate the camera to take zenithal or side images. Besides,
since the Raspberry Pi runs Linux and can be connected to the network, we can access it
through SSH protocol to remotely control it. Using the operating system task scheduling
tool, we take advantage of these conditions to schedule the image acquisition. Thus, we
only need to create the script to control the camera, and we can schedule its execution
easily. Nevertheless, the image acquisition process takes two phases: the image capture and
the pre-processing of that image.
Regarding the image capture phase, the next code shows the Python program capturing
the images.
11

Fig. 1. Camera setup in the greenhouse.
Listing 1.1. Python script that controls the camera
#!/usr/bin/python3
import time
from picamera2 import Picamera2, Preview
picam2 = Picamera2()
picam2.rotation = 180
picam2.vflip()
picam2.start_preview(Preview.NULL)
preview_config = picam2.create_preview_configuration(main={"size"
,→ :(5568,3132)}, buffer_count=2)
picam2.configure(preview_config)
picam2.start()
time.sleep(1)
picam2.set_controls({"AfTrigger": 1})
time.sleep(10)
date_time = time.strftime("%Y%m%d_%H%M%S")
image_filename = "image_"+date_time+".jpg"
picam2.capture_file(image_filename)
Once the image has been taken, the process’s second phase occurs. Since manually
resizing and cutting the images can produce bad samples, we decided to automate this
process. In addition, we discover that due to temperature changes inside the greenhouse,
the camera suffers little movement during the day and night, resulting in displaced images,
which makes it even more difficult to resize and cut manually. Thus, the second phase of
the process runs on the Raspberry Pi and uses mathematical functions like Homography,
HoughCircles, and the Python library OpenCV to clean, rectify, and cut the image as
needed.
Algorithm 1 shows the steps necessary to achieve that.
Regarding the two mathematical functions that intervene in the pre-processing of im-
ages, we use the Python library OpenCV that implements both functions.
HoughCircles is based on the Hough Transform, a technique originally developed to
detect lines inside an image. This function enables the detection of circles in an image
based on the Hough Transform, and the method proposed in [7]. One of the highlights of
12

Algorithm 1 Image pre-processing
Require:
Image I = captured_image
Number_of_rows r = n
Number_of_columns c = m
1: procedure Pre-process(I, r, c)
2: Make a temporal copy of the image temp=I
3: Convert temp to HSV color scale
4: Apply a color mask to temp with the color of reference points
5: Convert temp to gray-scale
6: Apply HoughCircles function to temp to get the coordinates of reference points in the image
coordinates={{x1, y1}, {x2, y2}, . . . {xn, yn}}
7: if coordinates mod 4 = 0 then
8: Apply the OpenCV Homography function by using coordinates points
9: Save the resulting image as rectified
10: if r > 0 AND c > 0 then
11: for i={0,. . . ,n} do
12: Cut the image rectified by i
13: Save the resulting image as rowi
14: for j={0,. . . ,n} do
15: Cut the image rowi by j
16: Save the resulting image as cell_i_j
17: end for
18: end for
19: end if
20: else
21: Log an error for image I
22: end if
23: end procedure
HoughCircles is its ability to efficiently handle circles with different sizes and noise levels
in the image, which in our case, is necessary to deal with.
When we pre-process the captured image, we first apply a mask to get only the content
similar to the color of the reference markers. Thus, we filter all the other elements in the
image to facilitate the recognition of the reference markers. Since the reference markers are
circle shaped, we can use the HoughCircles function to detect their x and y coordinates in
the image. Each image must have at least four reference markers. However, it is possible to
get more than four coordinate pairs. In those cases, we log an error to process the image
manually.
Homography is a geometric transformation that maps points in one image to corre-
sponding points in another, keeping the same center of projection and the alignment of
the elements in the image [8]. The homography method is the most general formulation
of linear transformations in projective geometry. For this reason, it is also known as a
projective transformation [9].
As shown in Fig. 2, the Homography is represented by a 3x3 matrix that relates points
in one image to their corresponding points in another. The homographic matrix is com-
puted from at least four corresponding points between the source and destination images,
employing least squares or RANSAC algorithms to obtain an accurate transformation for
various practical applications.
Our proposal uses the homography function to rectify the captured image before cutting
it into individual images.
After running the pre-processing phase, getting images containing individual samples
of the study objects is possible. These images differ from one research to another. For
example, if the analysis is focused on plant stems, it could be possible to get side images
13

Fig. 2. Homography planar projective transformation [8].
containing individual plants in pots. To do that, it is only required to place the reference
markers in the desirable position and wait for the result images.
It is also crucial to note that adjusting the name of the resulting images can also help
in the labeling of the samples. The proposed method covers only the capturing and pre-
processing of samples. Therefore, if needed, it is required to label the resulting images
manually.
4 Results
We conducted two study cases to evaluate the proposed method of capturing and pre-
processing image samples for smart farming. Following, we detail the study cases.
4.1 Study case 1: germination of plant growing in seedbeds
In this case, the objective is to determine the germination stage of Cannabis Sativa plants.
We attempt to create a classifier to recognize the state of plants in a seedbed in real time.
This way, we can easily know the percentage of germinated plants in the seedbed.
To achieve that, we need an image dataset of Cannabis Sativa plants germinating in
a seedbed at distinct growing stages. After a literature review, image datasets containing
samples of Cannabis Sativa germination have not been found. Although it is possible to get
image datasets containing plants for other species in the germination stage, those datasets
focus mainly on weed detection. Since we work in a greenhouse-controlled environment, we
seek to classify only images of the plants leaving aside the weed presence.
For this reason, we decided to create a new custom dataset. We work with seedbeds
where it is possible to plant 72 seeds simultaneously. But there was a problem related
to the processing of the captured images. Initially, we had to cut the images manually,
which caused bad samples. Secondly, we had to deal with every movement suffered by the
camera. Thus, every captured image must be manually rotated to align it and make cutting
possible. So, the whole process was too slow and stressful.
14

However, using the proposed method, we could create a clean and extensive dataset
containing more than 80000 images of 224x224 pixels, containing data of plants in different
germination stages. Fig. 3 shows an example of the captured image and the resulting
samples.
Fig. 3. Seedbed and samples obtained after running the proposed capturing and pre-processing method.
Currently, we are labeling the samples to train a deep-learning model and publish the
study results and the dataset. The proposed method was very easy to use because of the
automation of the script.
Having many plants in the seedbeds, we selected some and put them to grow in pots.
With these plants, we run another study case to analyze the growth of the plants. This
second study case will be detailed next.
4.2 Study case 2: modeling plants in different growing stages
For the second study case, we took side images of the plants in the pots. In this case, we
try to analyze the growing stages of the plants. The objective was to analyze individual
parts of the plants, like stems, leaves, or flowers.
From the literature [10], we can see that two open areas of research in smart farming
are related to using artificial intelligence techniques to count leaves, flowers, and fruits
automatically and the approximation of stem width through machine learning techniques.
We can contribute to that area by using deep learning techniques and an image dataset
containing samples of plants, leaves, flowers, fruits, and stems.
However, we suffer again from the scarcity of available training data. Although it is
possible to find datasets containing samples of plant parts for some crops, they are not
available for many other crops like Cannabis Sativa, making it necessary to work on the
creation of new custom datasets.
This way, we work on the creation of a second dataset. By knowing the distance between
the camera and the plants, it is possible to approximate the stem width. It is important to
15

highlight that to count leaves, flowers, and fruits; it is better to relocate the camera to the
zenithal position and take samples from that location. Hence, we created a second dataset
to analyze only the stem of plants.
Fig. 4. Sample obtained after running the proposed capturing and pre-processing method for stem analysis.
Fig. 4 shows an image after running the capturing and proposed pre-processing method.
For side images, it is crucial to place a uniform white wall or screen behind the plants. The
uniform color helps to get images where the plant is highlighted, and there is not any other
distraction altering the resulting image. It is also required to place the reference markers
in a position that makes it possible to capture the whole plant. It is important to note that
at the beginning, there could be a lot of blank space in the image. However, this way, we
can get a dataset of comparable images, making it possible to generate a time-lapse of the
plant growing.
Moreover, if we plan to analyze the stem growth and width, it is required to keep the
camera and the reference markers in the same position throughout the entire process. In
the proposed method of capturing and pre-processing, neither the camera nor the reference
markers move. In this case, we only have to guarantee keeping the position of the plant
pots.
Since we are analyzing plants’ growth, the second study case takes more time to be
completed. Nevertheless, we can approximate the size of the resulting dataset considering
the time period defined for the image capturing and the number of pots appearing in the
images. For the second study case, we defined an image capture every four hours from 8:00
to 16:00. This decision was made because we were analyzing the growth of the plants, so
we did not need images so closely in time. This way, we can get three daily images, each
containing four pots. After the pre-processing of the images, we got twelve samples every
day. Considering that the growing phase of the Cannabis plants can take approximately
four months, we can get a dataset composed of 1440 image samples.
16

From the two executed study cases, we want to highlight that the resulting image sam-
ples are individual and unique. We do not recur to augmentation techniques. We only pre-
process the original captured images following the proposed method. Besides, our method
enables us to get clean and high-quality datasets, where the samples’ size is usually bigger
than the needed by deep learning models, making it possible to re-scale the images as
needed.
In the next section, we present the conclusions resulting from this work. We also state
some ideas we consider can be done in future work.
5 Conclusions and future work
The scarcity of datasets for training deep learning models highlights the importance of
creating initiatives to share and release public datasets, encourage research collaboration,
and establish more efficient and consistent data collection and labeling practices. To deal
with scarcity, creating new high-quality training datasets is necessary. Nevertheless, this
task is very hard to achieve because it requires to guarantee that datasets accomplish size,
resolution, and volume adequate for complex models.
We consider that automating the capture and preprocessing steps can reduce the effort
required to create new datasets, enabling the researchers to focus on other important
activities like labeling and creating the models.
In this paper, we proposed a method to ease the creation of image datasets for smart
farming applications. The proposed method enables capturing and preprocessing images
automatically and periodically. This way, getting a large volume of images from an indi-
vidual setup is possible.
With the proposed method, we contribute to creating new datasets for training smart
farming AI models. During this research, we conducted two study cases in which we can
use the proposed method to create new datasets. First, the germination dataset is currently
being used to train a deep-learning model to identify the germination of Cannabis Sativa
plants in seedbeds in real-time. Meanwhile, a second study case produces a dataset con-
taining images of plants at different stages of growth. This dataset will be used to analyze
the plants’ stem width and create a model of the growing plants.
The results obtained from using the proposed method are very positive since we try to
manually execute the capturing and pre-processing of images, resulting in a stressful and
slow task. With the proposed method, creating the dataset while doing other activities is
possible.
Besides, the proposed method enables the researcher to adjust the name of the resulting
images, easing the labeling phase of the dataset creation. By selecting the correct name
for the resulting images, it can be possible to get datasets partially labeled, reducing the
work of labeling tasks.
Creating new training datasets is an ever-evolving, collaborative effort involving data
scientists, researchers, and engineers. By investing in the generation of high-quality and
relevant data, we make possible the development and continuous improvement of deep
learning models that can overcome complex challenges and offer innovative solutions for
the advancement and benefit of society.
We plan to use the proposed method outside the greenhouse in future work. In this case,
the most challenging task is to place and keep the reference markers in the same position.
However, we consider that our proposal could be very helpful with crop production in
outdoor environments.
17

Acknowledgements
This work was partially supported by the Research Center for Communication and Infor-
mation Technologies (CITIC) and the School of Computer Science and Informatics (ECCI)
at the University of Costa Rica, Research Project No. 834-B9-189. This study is part of
the AGROALNEXT program (AGROALNEXT/2022/048) and has been supported by
MCIN with funding from the European Union NextGenerationEU (PRTR-C17.I1) and the
Generalitat Valenciana.
References
1. P. Villalobos, J. Sevilla, L. Heim, T. Besiroglu, M. Hobbhahn, and A. Ho, “Will we run out of
data? an analysis of the limits of scaling datasets in machine learning,” 2022. [Online]. Available:
https://arxiv.org/abs/2211.04325
2. Y. Lu and S. Young, “A survey of public datasets for computer vision tasks in precision agricul-
ture,” Computers and Electronics in Agriculture, vol. 178, p. 105760, Nov. 2020. [Online]. Available:
https://doi.org/10.1016/j.compag.2020.105760
3. M. J. Pappaterra, F. Flammini, V. Vittorini, and N. Besinovi ˇ c, “A ´ systematic review of artificial
intelligence public datasets for railway applications,” Infrastructures, vol. 6, no. 10, p. 136, Sep. 2021.
[Online]. Available: https://doi.org/10.3390/infrastructures6100136
4. L. Alzubaidi, J. Bai, A. Al-Sabaawi, J. Santamar´ıa, A. S. Albahri, B. S. N. Al-dabbagh, M. A. Fadhel,
M. Manoufali, J. Zhang, A. H. Al-Timemy, Y. Duan, A. Abdullah, L. Farhan, Y. Lu, A. Gupta, F.
Albu, A. Abbosh, and Y. Gu, “A survey on deep learning tools dealing with data scarcity: definitions,
challenges, solutions, tips, and applications,” Journal of Big Data, vol. 10, no. 1, Apr. 2023. [Online].
Available: https://doi.org/10.1186/s40537-023-00727-2
5. D. Bajpai and L. He, “Custom dataset creation with tensorflow framework and image processing for
google t-rex,” in 2020 12th International Conference on Computational Intelligence and Communication
Networks (CICN), 2020, pp. 45–48.
6. R. R. Barioni, W. L. Costa, J. A. C. Neto, L. S. Figueiredo, V. Teichrieb, J. P. Quintino, F. Q. B.
da Silva, A. L. M. Santos, and H. Pinho, “Hutrain: a framework for fast creation of real human pose
datasets,” in 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-
Adjunct), 2020, pp. 65–66.
7. H. Yuen, J. Princen, J. Illingworth, and J. Kittler, “Comparative study of hough transform methods
for circle finding,” Image and Vision Computing, vol. 8, no. 1, pp. 71–77, Feb. 1990. [Online]. Available:
https://doi.org/10.1016/0262-8856(90)90059-e
8. R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge
University Press, ISBN: 0521540518, 2004.
9. R. Zeng, “Homography estimation: From geometry to deep learning,” Ph.D. dissertation. [Online]. Avail-
able: https://doi.org/10.5204/thesis.eprints.134132
10. J. Brenes, A. Mart´ınez, C. Quesada-Lopez, and M. Jenkins, ´ “Decision support systems that use
artificial intelligence for precision agriculture: A systematic literature mapping [sistemas de apoyo a la
toma de decisiones que usan inteligencia artificial en la agricultura de precision: Un mapeo sistem ´ atico
de literatura],” ´ RISTI - Revista Iberica de Sistemas e Tecnologias de Informacao, vol. 2020, no. E28,
pp. 217–229, 2020, cited By 0. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-
s2.0- 85081021329partnerID=40md5=a6d1e4a4fc59451683aec961b6576192
Authors
Jose A. Brenes received his Master in Computer Science from the University of Costa
Rica. He did a degree in computer science at the University of Costa Rica. He is pursu-
ing his PhD in Computer Science from the University of Costa Rica and the University
18

of Alicante. His research interests include Smart farming, Computer networks, Human-
Computer Interaction, and Information Technology Management.
Gabriela Marín-Raventós received a MSc in Computer Science from Case Western
Reserve University and a Ph.D. in Business Analysis and Research from Texas A&M Uni-
versity, United States. She did a degree in Computer Science at the University of Costa
Rica. His research interests include Smart Cities, Human-Computer Interaction, Decision
Support Systems, Gender in IT, and Digital Equity.
Javier Ferrández-Pastor received Ph.D. from the University of Alicante, Spain. He did
a degree in Industrial Engineering at the University of Oviedo, Spain. His research inter-
ests include Information, communication and control technologies, Monitoring and control
applications, and Artificial Intelligence.
José M. Cámara-Zapata received Ph.D. from the University Miguel Hernández of Elche, Spain.
He did a degree in Agricultural Engineering at the Valencia Polytechnic University, Spain. His
research interests include Monitoring, Modeling, and Control applications.
19

Use of Hough Transform and Homography for the Creation of Image Corpora for Smart Agriculture

More Related Content

Similar to Use of Hough Transform and Homography for the Creation of Image Corpora for Smart Agriculture (20)

Recently uploaded (20)

Use of Hough Transform and Homography for the Creation of Image Corpora for Smart Agriculture