SlideShare a Scribd company logo
Minor Project Report on
Machine Vision- Application on Embedded Platform
Submitted by,
Omkar Rane T187014
Abhishek Sainkar T187009
Kaustubh Wankhede T187003
Guided by,
Prof. Bhairavi Savant
A Report submitted,
In partial fulfilment of the requirements for Sixth Semester of
BACHELOR OF TECHNOLOGY in
ELECTRONICS & TELECOMMUNICATION ENGG.
School of Electrical Engineering
Alandi (D), Pune – 412105
2018-19
School of Electrical Engg.
UG Program in E&TC Engg
(Accredited by NBA, ISO 9001:2015 Certified)
CERTIFICATE
This is certify that,
Omkar Rane T187014
Abhishek Sainkar T187009
Kaustubh Wankhede T187003
of T.Y. B.Tech. have submitted a Report on,
Machine Vision- Application on Embedded Platform
The said work is completed by putting the requirement of hours as per prescribed curriculum
during the academic year 2018 – 19. The report is submitted in the partial fulfilment of the
requirements for the course Mini Project in the Sixth Semester of Degree of Engineering in -
Electronics and Telecommunication Engineering of MIT Academy of Engineering.
Prof. Bhairavi Savant Dr. Debashis Adhikari
Project Advisor Dean, SEE
External Examiner
Contents
Abstract i
Acknowledgement ii
List of Figures iii
List of Tables iv
1. Introduction 1
1.1 Motivation 1
1.2 Objectives and Scope 2
1.3 Problem Statement 3
2. Literature Survey 4
3. System Design 5
3.1 Hardware Requirements 5
3.2 Software Requirements 7
3.3 Methodology 8
4. Implementation Details 16
5. Results 22
6. Conclusion and Future scope 26
Project Plan
Bill of Material
References
i
Abstract
Our Project mainly focuses on the Machine Vision Application. We have deployed the color
segmentation algorithm to detect the objects integrated with embedded board. The Project focuses
on image-based object detection technique to recognize the object. We have successfully identified
the objects. Our work would be implemented in automatic inspection, process control and robot
guidance usually in Industry 4.0
ii
Acknowledgement
We are thankful to all the people who are involved in this project including jury members who
suggested rectification of errors in our project from time to time during our presentations. We are
also thankful to School of Electrical Engineering who had helped in project. Also, the Guide
Prof. Bhairavi Savant for Project by their guidance we were doing the appropriate changes. We
are also thankful to Prof. Smita Kulkarni for their guidance We would like to thank to all who
knowingly and unknowingly helped us to accomplish this project of third year in sixth semester.
Omkar Rane
Abhishek Sainkar
Kaustubh Wankhede
iii
iii
List of Figures
Fig. No. Fig. Name Page No.
Fig 3.1 Raspberry Pi 3 Block Diagram 5
Fig 3.2 Raspi Camera 6
Fig 3.3 Block Diagram for Detection 8
Fig 3.4 Raspberry Pi configuration 9
Fig 3.5 Setting upper and lower limit of each colour in HSV. 14
Fig 4.1 HSV Colour Model 18
Fig 4.2 Camera Interfacing with Raspberry Pi 21
Fig 5.1 Compilation of Python file in Shell 22
Fig 5.2 Detection of Red Bottle 22
Fig 5.3 Detection of Yellow Bottle 23
Fig 5.4 Detection of Green Bottle 23
Fig 5.5 Colour detection setup 24
Fig 5.6 Detection of Blue Bottle 24
Fig 5.7 Detection of Light Blue Bottle 25
Fig 5.8
Detection of Dark Blue Bottle
25
iv
List of Tables
Table. No. Table. Name Page No.
Table 3.1 Raspberry Pi Specification 6
Table 3.2 PiCamera Specification 7
Page | 1
Chapter 1
Introduction
1.1 Motivation
For a few decades, computer scientists and engineers have attached cameras and simplistic image
interpretation methods to a computer (robot) in order to impart vision to the machine. A lot of
interest has been shown towards object recognition, object detection, object categorization etc.
Simply speaking, object recognition deals with training the computer to identify a particular
object from various perspectives, in various lighting conditions, and with various backgrounds;
object detection deals with identifying the presence of various individual objects in an image;
and object categorization deals with recognizing objects belonging to various categories. For
example, a domestic help robot can be trained to recognize if an object is a coffee machine
(object recognition), it may be trained to detect a coffee machine in the kitchen (object
detection), and it may be trained to identify cups of various types and forms into a common
category called cups.[01]
Despite the simplistic definition mentioned above, the lines separating the three skills above are
very blur and the problems often intermingle in terms of the challenges as well as solution
approaches. Further, it is evident that for practical purposes, a good combination of all the three
skills is essential. Great success has been achieved in controlled environment for object
detection/recognition problem but the problem remains unsolved in uncontrolled places, in
particular, when objects are placed in arbitrary poses in cluttered and occluded environment. As
an example, it might be easy to train a domestic help robot to recognize the presence of coffee
machine with nothing else in the image. On the other hand, imagine the difficulty of such 2 robot
in detecting the machine on a kitchen slab that is cluttered by other utensils, gadgets, tools, etc.
The searching or recognition process in such scenario is very difficult. So far, no effective
solution has been found for this problem. Despite a lot of research in this area, the methods
developed so far are not efficient, require long training time, are not suitable for real time
Page | 2
application, and are not scalable to large number of classes. Object detection is relatively simpler
if the machine is looking for detecting one particular object (say coffee machine). However,
recognizing all the objects inherently requires the skill to differentiate one object from the other,
though they may be of same type. Such problem is very difficult for machines, if they do not
know about the various possibilities of objects.[02]
Besides intra-class and inter-class separation problem, another big problem is how many
categories should be considered and what should be the basis for recognizing a category, forming
a new category, etc. This also involves the problem of enabling the machine to classify objects in
existing categories, if possible, recognizing the objects that do not belong to any existing
categories, and learning new categories. For object categorization, there is no exhaustive and
exclusive set of objects, which can be used to train the machine.[03]
1.2 Objectives and Scope
Scope:
For mini project we are restricting ourselves to detection of objects by camera by its color.
Object detection is one of major step in machine vision. Given a set of object classes, object
detection consists in determining the location and scale of all object instances, if any, that are
present in an image. Thus, the objective of an object detector is to find all object instances of one
or more given object classes regardless of scale, location, pose, view with respect to the camera,
partial occlusions, and illumination conditions.
Objective:
➢ Detection of all objects in scope of vision.
➢ Gather Data in form of images.
➢ Train Model for detection of objects.
➢ Segmentation of objects according to user need.
Page | 3
1.3 Problem Statement
To design and Machine Vision application on embedded platform based on ARM Cortex
processor and Embedded Linux operating system and using color segmentation algorithm for
detection of objects based upon its color via OpenCV Machine Learning Libraries. Our Aim to
detect particular object based upon color combination chosen by user.
Page | 4
Chapter 2
Literature Survey
1. O. Alhusain[05] discuss that In the food industry there are problems such as The first
one is the decline in food quality, and the second one is the “waste” problem associated
with processing and preparation operations. Hence, there is the need for quality
inspection and assurance mechanisms to be installed in the production lines of such mass
food processing.
2. Jarimopas et al. [06] have described development a machine vision experimentally
sorting sweat tamarinds parameter shape, size, colour and defects. It was performed with
image processing software analysis the image and hardware design include a conveyor,
control drive light source, image antenna, manage unit and micro-computer.
3. Tushar Jain [07] described Machine vision provides innovative solutions in the direction
of industrial automation. These activities include, among others, delicate electronics
component manufacturing, quality textile production, metal product finishing, glass
manufacturing, machine parts, printing products and granite quality inspection, integrated
circuits manufacturing and many others. Machine vision technology improves
productivity and quality management and provides a competitive advantage to industries
that employ this technology.
4. Industries based on automation, consumer markets, medical domains, Défense and
surveillance sectors are most likely domains extensively using machine vision. Image
classification, being the widely researched area in the domain of computer vision has
achieved remarkable results in world-wide competitions.
Page | 5
Chapter 3
System Design
3.1 Hardware Requirements:
1) Raspberry Pi 3 b
Raspberry Pi single-board computer represent this concept as affordable educational
platform. Here at xDevs.com various versions of Raspberry Pi are already widely used as
test equipment controllers and live datalogging platform. Low cost (around $40 USD for
fully-featured Raspberry Pi 3) and compact size allow easy adding of standalone modules
for running different experiments, and invisible to total project budget since involved test
equipment often costs many times more.[02]
Fig 3.1 Raspberry Pi 3 Block Diagram
Page | 6
Heart of the Pi is the Broadcom BCM2837 SOC which has quad-core Cortex-A53 ARM
processor and VideoCore IV GPU. As these implementations are proprietary to
Broadcom, we don’t know exact details, however for purpose of overclocking it’s less
important. GPU supports OpenGL ES standard and capable to decode Full-HD 60fps
video content in real-time. Separate 1GB LPDDR2 SDRAM memory chip
EDB8132B4PB-8D-F from Elpida (Micron) located on the bottom side.
Table 3.1 Raspberry Pi Specification
2. Raspi Camera V1
The Raspberry Pi Camera Board plugs directly into the CSI connector on the Raspberry Pi. It's
able to deliver a crystal clear 5MP resolution image, or 1080p HD video recording at 30fps.
Fig 3.2 Raspi Camera
Page | 7
Table 3.2 PiCamera Specification
3.2 Software Requirements
OpenCV:
OpenCV (Open Source Computer Vision) is a community driven open source software library,
making it a good place to start working with computer vision. An obvious drawback with using
this library is the fact that there is no official IDE to use when building applications. To build an
application the user is required to use one of the programming languages mentioned in the
paragraph below, making this the tool with the steepest learning curve. Due to the immense user
base there are however, many forums in which problems and new ideas can be discussed,
making the lack of an IDE less of a problem. Written in C++ the library is accessible using C,
C++, Python and Java adding to the availability and possibilities when using this library.
Furthermore, OpenCV supports most cameras and frame grabbers. The problem that could arise
concerning these is: driver compatibility with the computer's operating system. This is however,
a possible problem which concerns all three of the libraries in question. The software library is
registered as a BSD licence meaning the modification and copying of the source code is allowed
Page | 8
without having to share the results with the community. Therefore, it is free to use in
commercial, industrial and private projects.[10]
OpenCV is written in C++ and its primary interface is in C++, but it still retains a less
comprehensive though extensive older C interface. There are bindings in Python, Java and
MATLAB/OCTAVE. The API for these interfaces can be found in the online documentation.
Wrappers in other languages such as C#, Perl, and Ruby have been developed to encourage
adoption by a wider audience. OpenCV runs on the following desktop operating systems:
Windows, Linux, macOS, FreeBSD, NetBSD, OpenBSD. OpenCV runs on the following mobile
operating systems: Android, iOS, Maemi, Blackberry The user can get official releases from
Source Forge or take the latest sources from GitHub.
3.3 Methodology
Fig 3.3 Block Diagram for Detection
Raspberry Pi Camera Installation:
Install the camera module by plugging the camera wire into the port labelled camera, which is
next to the HDMI port on the raspberry pi. Once installed update and upgrade as seen below
sudo apt-get update
sudo apt-get upgrade
Next you must enable the camera, this can be done by going into the "Applications Menu", and
under "Preferences" select "Raspberry Pi Configuration". Under the "Interfaces" tab press the
"Enable" radio button for Camera. Remotely this can be done using "sudo raspi-config", and in
"Interfacing Options" enable the camera.[03]
Page | 9
Fig 3.4 Raspberry Pi configuration
Installation Of OpenCV:
OpenCV is an open source computer vision and machine learning software library. OpenCV is
released under a BSD license making it free for both academic and commercial use. It has
C++, Python, and Java interfaces and supports Windows, Linux, Mac OS, iOS, and Android.
OpenCV was designed for computational efficiency and a strong focus on real-time applications.
To install OpenCV, we need to have Python installed. Since Raspberry Pis are preloaded with
Python, we can install OpenCV directly.
Type the commands below to make sure your Raspberry Pi is up to date and to update the
installed packages on your Raspberry Pi to the latest versions.[03]
sudo apt-get update
Page | 10
sudo apt-get upgrade
Type the following commands in the terminal to install the required packages for OpenCV on
your Raspberry Pi.
sudo apt install libatlas3-base libsz2 libharfbuzz0b libtiff5 libjasper1 libilmbase12 libopenexr22
libilmbase12 libgstreamer1.0-0 libavcodec57 libavformat57 libavutil55 libswscale4 libqtgui4
libqt4-test libqtcore4
Type the following command to install OpenCV 3 for Python 3 on your Raspberry Pi, pip3 tells
us that OpenCV will get installed for Python 3.
sudo pip3 install opencv-contrib-python libwebp6
1. Import Packages
First, import the packages required for this project.
import cv2
from picamera.array import PiRGBArray
from picamera import PiCamera
import numpy as np
Page | 11
2. Create Your Trackbars
Next, we create the trackbars to help us with selecting a color. To create the trackbars, we have
the cv2.createTrackbar() function. We have five arguments to create:
1. The trackbar name
2. The window name to which it is attached
3. The default values
4. The maximum value
5. The call back function executed every time trackbar value changes
The callback function always has a default argument, which is the trackbar position In our case,
the function does nothing, so we will simply pass.
We are going to create three trackbars for blue, green, and red. Each trackbar will have a default
value of 0 and a maximum value of 255 and will be attached to the window named Trackbars
def nothing(x):
pass
cv2.namedWindow("Trackbars")
cv2.createTrackbar("B", "Trackbars", 0, 255, nothing)
cv2.createTrackbar("G", "Trackbars", 0, 255, nothing)
cv2.createTrackbar("R", "Trackbars", 0, 255, nothing)
More
Page | 12
3. Setting up the Raspberry Pi Camera
Now, we can initialize the camera object that allows us to play with the Raspberry Pi camera.
We set the resolution at (640, 480) and the frame rate at 30 fps
camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 30
4. The PiRGBArray()
PiRGBArray() gives us a 3-dimensional RGB array organized (rows, columns, colors) from an
unencoded RGB capture.
PiRGBArray gives us the advantage of reading the frames from Raspberry Pi camera as NumPy
arrays, making it compatible with the OpenCV. It avoids the conversion from JPEG format to
OpenCV format which would slow our process.
PiRGBArray()takes 2 arguments: the first is the camera object and the second is the resolution.
rawCapture = PiRGBArray(camera, size=(640, 480)
5. The capture continuous Function
Now we are going to use the capture_continuous function to start reading the frames from the
Raspberry Pi camera module.The capture_continuous function takes three arguments:
1) rawCapture
Page | 13
2) The format in which we want to read each frame. Since OpenCV expects the image to be
in the BGR format rather than the RGB, we need to specify the format to be BGR.
3) The use_video_port boolean. Making this true means that we treat the stream as video.
Once we have the frame, we can access the raw NumPy array via the .arrayattribute.
for frame in camera.capture_continuous(rawCapture, format="bgr",
use_video_port=True):
image = frame.array
6. Setting up the Color Recognition
Now we are going to convert images from the BGR to the HSV color space. The HSV (hue
saturation value) space gives us better results when doing color-based segmentation.
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV
Next, we create the trackbars used to select the color.
B = cv2.getTrackbarPos("B", "Trackbars")
G = cv2.getTrackbarPos("G", "Trackbars")
R = cv2.getTrackbarPos("R", "Trackbars")
After that, we can find out the lower and upper limit of the color in HSV.
green = np.uint8([[[B, G, R]]])
hsvGreen = cv2.cvtColor(green,cv2.COLOR_BGR2HSV)
Page | 14
lowerLimit = np.uint8([hsvGreen[0][0][0]-10,100,100])
upperLimit = np.uint8([hsvGreen[0][0][0]+10,255,255])
Fig 3.5 Setting upper and lower limit of each colour in HSV.
Next, we adjust the threshold of the HSV image for a range of each selected color.
mask = cv2.inRange(hsv, lowerLimit, upperLimit)
Now we can extract the objects of the colors in the frame.
result = cv2.bitwise_and(image , image , mask=mask)
Let’s show the result in the output window. cv2.waitkey() is a keyboard binding function and
waits for a specified amount of time for any keyboard event. It takes only one argument, which is
set to time in milliseconds. If the key is pressed in that time then the program will continue.
Passing 0 means it will wait indefinitely for a key.
cv2.imshow("frame", image)
Page | 15
cv2.imshow("mask", mask)
cv2.imshow("result", result)
key = cv2.waitKey(1)
Always clear the stream in preparation for the next frame by calling truncate (0) between
captures.
rawCapture.truncate(0)
Page | 16
Chapter 4
Implementation Details
1. Image Segmentation
The process of partitioning a digital image into multiple segments is defined as image
segmentation. Segmentation aims to divide an image into regions that can be more
representative and easier to analyze. Such regions may correspond to individual surfaces,
objects, or natural parts of objects. Typically, image segmentation is the process used to
locate objects and boundaries (e.g., lines or curves) in images. Furthermore, it can be
defined as the process of labeling every pixel in an image, where all pixels having the
same label share certain visual characteristics. Usually segmentation uses local
information in the digital image to compute the best segmentation, such as color
information used to create histograms or information indicating edges, boundaries, or
texture information [10].
Color image segmentation that is based on the color feature of image pixels assumes that
homogeneous colors in the image correspond to separate clusters and hence meaningful
objects in the image. In other words, each cluster defines a class of pixels that share
similar color properties. As the segmentation results depend on the used color space,
there is no single-color space that can provide acceptable results for all kinds of images.
For this reason, many authors tried to determine the color space that will suit their
specific color image segmentation problem. In this work, a segmentation of color images
is tested with different classical color spaces, RGB, CMY, XYZ & HSV to select the best
color space for the considered kind of images.[10]
The segmentation process is based on the Grab Cut segmentation technique, which is
considered as one of the powerful state-of-the-art techniques for the problem of color
image segmentation. The iterative energy minimization scheme of the Grab Cut is based
on the powerful optimization of the Graph Cut technique which allows for the generation
of the global optimal segmentation. In addition, Graph Cut can be easily well extended to
the problem of N-D images. Furthermore, the cost energy function of the Graph Cut
Page | 17
minimization process allows it to be defined in terms of different image features such as
color, region, boundary, or any mixture of image features. This flexibility provides wide
potential for the use of Grab Cut in different applications. On the other hand, GrabCut is
considered as a billable segmentation technique, where images can be segmented into
two background and foreground regions only. Initial user intervention is required in order
to specify an object of interest to be segmented out of the image, considering all the
remaining image pixels as one background region. This classifies the GrabCut as a
semiautomatic segmentation technique and turns the quality of the initialization and
hence the segmentation performance, sensitive to the user selection. In other words, poor
GrabCut initialization may lead to bad final segmentation accuracy which might require
extra user interactions with the segmentation results for fine tuning
How does Image Segmentation work?
We can divide or partition the image into various parts called segments. It’s not a great
idea to process the entire image at the same time as there will be regions in the image
which do not contain any information. By dividing the image into segments, we can make
use of the important segments for processing the image. That, in a nutshell, is how image
segmentation works.
An image is a collection or set of different pixels. We group together the pixels that have
similar attributes using image segmentation. Take a moment to go through the below
visual (it’ll give you a practical idea of image segmentation.[10]
Object detection builds a bounding box corresponding to each class in the image. But it
tells us nothing about the shape of the object. We only get the set of bounding box
coordinates. We want to get more information – this is too vague for our purposes.
Image segmentation creates a pixel-wise mask for each object in the image. This
technique gives us a far more granular understanding of the object(s) in the image.
Why do we need to go this deep? Can’t all image processing tasks be solved using simple
bounding box coordinates? Let’s take a real-world example to answer this pertinent
question.
Page | 18
2. The hue-saturation-value (HSV) color model
Another way to characterize a color is in terms of the HSV model.
•The hue (H) of a color refers to which pure color it resembles. All tints, tones and shades of red
have the same hue.
Hues are described by a number that specifies the position of the corresponding pure color on the
color wheel, as a fraction between 0 and 1. Value 0 refers to red; 1/6 is yellow; 1/3 is green; and
so forth around the color wheel.
•The saturation (S) of a color describes how white the color is. A pure red is fully saturated, with
a saturation of 1; tints of red have saturations less than 1; and white has a saturation of 0.
•The value (V) of a color, also called its lightness, describes how dark the color is. A value of 0
is black, with increasing lightness moving away from black.
This diagram, called the single-hex cone model of color space, can help you visualize the
meaning of the H, S, and V parameters.
Fig 4.1 HSV Colour Model
Page | 19
•The outer edge of the top of the cone is the color wheel, with all the pure colors. The H
parameter describes the angle around the wheel.
•The S (saturation) is zero for any color on the axis of the cone; the center of the top circle is
white. An increase in the value of S corresponds to a movement away from the axis.
•The V (value or lightness) is zero for black. An increase in the value of V corresponds to a
movement away from black and toward the top of the cone.
The Ostwald diagram corresponds to a slice of this cone. For example, the triangle between red,
white, and black is the Ostwald diagram for the varieties of red.
3. RGB to HSV conversion formula
The R, G, B values are divided by 255 to change the range from 0..255 to 0..1:
R' = R/255
G' = G/255
B' = B/255
Cmax = max(R', G', B')
Cmin = min(R', G', B')
Δ = Cmax – Cmin
Hue calculation:
Page | 20
Saturation calculation:
Value calculation:
V = Cmax
4. Algorithm
Step 1: Run shell Command python proj.py
Step 2: Import OpenCV2 library as CV2
Step 3: Import Numpy Library as np
Step 4: from picamera.array import PiRGBArray
Step 5: from picamera import PiCamera
Step 6: Create trackbar cv2.Trackbar() for R,G,B for colour Segmentation values.
Step 7: Set Resolution of camera and frame rate for image specification.
camera.resolution = (640, 480)
camera.framerate = 30
Step 8: Capture Continuous Frames in Camera for Image Acquisition.
Step 9: Do Background Subtraction.
Step 10: Convert Red, Green and Blue to Hue Saturation Value.
Step 11: get lower and upper limit value from HSV values.
Step 12: Mask the values received from lower and upper limit of HSV.
Step 13: Close window.
Page | 21
5. Raspberry Pi with Camera
Fig 4.2 Camera Interfacing with Raspberry Pi
Page | 22
Chapter 5
Results
Fig 5.1 Compilation of Python file in Shell
Fig 5.2 Detection of Red Bottle
Page | 23
Fig 5.4 Detection of Green Bottle
Fig 5.3 Detection of Yellow Bottle
Page | 24
Fig 5.5 Colour detection setup
Fig 5.6 Detection of Blue Bottle
Page | 25
Fig 5.8 Detection of Dark Blue Bottle
Fig 5.7 Detection of light Blue Bottle
Page | 26
Chapter 6
Conclusion and Future Scope
In this project, an attempt was made to develop an object detection and tracking framework able
to run in real-time on a Raspberry Pi 3 Model B. By using colour segmentation method, we were
able to classify objects. This Project has future scope for implementation such detection for
industrial Automation and quality control practices.
OpenCV as the vision library of choice, is more than large and powerful enough to build a
program of this type. The documentation supporting the library is well written and easy to
understand. The wealth of examples and explanations in the OpenCV community have been able
to answer any and all questions that arose during the development process, and has been
invaluable as a source of knowledge. The program has been developed, exclusively, using open
source software. This points to the fact that further development would most likely lead to a
stable and cheap solution that could be incorporated into a commercial product, with no attached
licensing fees. The argument can also be made, concerning the hardware, that the Raspberry Pi,
while being a good platform during development, is too costly as the ultimate hardware in
production. The hardware performance can definitely be lower, than what the Raspberry Pi
supplies, and still perform at a satisfying speed. Thus, cutting the cost of a final product even
further.
In the future, we would make Real time object detection which will be able to distinguish them,
also count the number of objects in front.
PROJECT PLAN
Sr.
No
Week Work to Be Done Remark
1 1st Deciding of Topic Completed
2 2nd Literature Survey Completed
3 3rd Installing OS on Raspberry Pi Completed
4 4th & 5th Raspberry Pi Configuration Completed
5 6th Purchased Camera Completed
6 7th Configure camera with raspberry Pi Completed
7 8th
& 9th Install OpenCV on Raspberry Pi Completed
8 10th Installed Supportive libraries of OpenCV Completed
9 11th
& 12th Python Coding Completed
10 13th Image Segmentation Completed
11 14th
& 15th Testing Completed
12 16th Result Collection Completed
13 17th
& 18th Report Writing Completed
BILL OF MATERIAL
Sr no Component Price (Rs)
1 Raspberry Pi 3 Model B 3500
2 PICAMERA REES52 800
Total: Rs. 4300
REFERENCES
[ 01 ] Bradski, G., Kaehler A. Learning OpenCV: Computer Vision in C++ with the OpenCV
Library. Sebastopol, California : O’Reilly Media, Inc., 2008.
[ 02 ] http://eie.uonbi.ac.ke/sites/default/files/cae/engineering/eie/Computer%20Vision%20thro
ugh %20the%20Raspberry%20PI%20Counting%20Objects.pdf
[ 03 ] https://www.pyimagesearch.com/2018/09/26/install-opencv-4-on-your-raspberry-pi/
[ 04 ]
https://medium.com/nanonets/how-to-easily-detect-objects-with-deep-learning-on-
raspberrypi-225f29635c74
[ 05 ]
O. Alhusain, VISION-BASED SYSTEM FOR QUALITY CONTROL OF SOME FOOD
PRODUCTS. Wilson.,2003
[ 06 ] Jarimopas, Nitipong Jaisin, An experimental machine vision system for sorting sweet
tamarind.. 2008, Elsevier
[ 07 ] Tushar Jain, Meenu. Automation and Integration of Industries through Computer Vision
Systems..2013, IRPHOUSE
[ 08 ] What is a Raspberry Pi?”. Raspberry Foundation, 2015. [Online]. Available:
https://www.raspberrypi.org/help/what-is-a-raspberry-pi/.
[ 09 ] https://www.rs-online.com/designspark/object-tracking-using-computer-vision-and-
raspberry-pi
[ 10 ] https://www.hindawi.com/journals/tswj/2014/126025/

More Related Content

PDF
Dsip and aisc syllabus
PDF
IRJET - Facial Recognition based Attendance Management System
PDF
IRJET - Visual E-Commerce Application using Deep Learning
PDF
IRJET- Survey on Various Techniques of Attendance marking and Attention D...
PDF
P-D controller computer vision and robotics integration based for student’s p...
DOCX
MAJOR PROJECT
DOCX
PDF
Project Clearn: Studies in application of augmented reality in E Learning Cou...
Dsip and aisc syllabus
IRJET - Facial Recognition based Attendance Management System
IRJET - Visual E-Commerce Application using Deep Learning
IRJET- Survey on Various Techniques of Attendance marking and Attention D...
P-D controller computer vision and robotics integration based for student’s p...
MAJOR PROJECT
Project Clearn: Studies in application of augmented reality in E Learning Cou...

What's hot (8)

PDF
Embedded System Practicum Module for Increase Student Comprehension of Microc...
PPTX
Ioe orientation
PDF
MULTIMODAL BIOMETRICS RECOGNITION FROM FACIAL VIDEO VIA DEEP LEARNING
PDF
IRJET - Expiry Date and Cost Tracking in Medicine for Visually Impaired
PDF
Reusability Metrics for Object-Oriented System: An Alternative Approach
PDF
An Iot Based Smart Manifold Attendance System
PDF
Mobile augmented reality using 3D ruler in a robotic educational module to pr...
PDF
Face Recognition and Increased Reality System for Mobile Devices
Embedded System Practicum Module for Increase Student Comprehension of Microc...
Ioe orientation
MULTIMODAL BIOMETRICS RECOGNITION FROM FACIAL VIDEO VIA DEEP LEARNING
IRJET - Expiry Date and Cost Tracking in Medicine for Visually Impaired
Reusability Metrics for Object-Oriented System: An Alternative Approach
An Iot Based Smart Manifold Attendance System
Mobile augmented reality using 3D ruler in a robotic educational module to pr...
Face Recognition and Increased Reality System for Mobile Devices
Ad

Similar to Machine Vision On Embedded Platform -Report (20)

PDF
IRJET- Object Detection and Recognition for Blind Assistance
PDF
IOT based Automatic Colour Sorting Machine
PDF
HANDWRITTEN DIGIT RECOGNITION USING MACHINE LEARNING
PDF
HANDWRITTEN DIGIT RECOGNITION USING MACHINE LEARNING
PDF
Machine vision Application
PDF
Machine Vision On Embedded Platform
DOCX
Multiple object detection report
PDF
Electronic Circuit Assessment using Machine Learning (ML)
PDF
Real Time Moving Object Detection for Day-Night Surveillance using AI
PPTX
What is machine vision slide share
PDF
Design, Analysis and Fabrication of Pick & Place Colour Sorting Robotic Arm
PDF
IRJET- Analysing Wound Area Measurement using Android App
PDF
IRJET- Automated Student’s Attendance Management using Convolutional Neural N...
PDF
Human pose detection using machine learning by Grandel
PDF
IRJET - Human Pose Detection using Deep Learning
PDF
docmentation with rules
DOC
Face Detection And Tracking
PDF
A SOFTWARE REQUIREMENT ENGINEERING TECHNIQUE USING OOADA-RE AND CSC FOR IOT B...
DOCX
Internship report on AI , ML & IIOT and project responses full docs
PDF
Automated Attendance Management System
IRJET- Object Detection and Recognition for Blind Assistance
IOT based Automatic Colour Sorting Machine
HANDWRITTEN DIGIT RECOGNITION USING MACHINE LEARNING
HANDWRITTEN DIGIT RECOGNITION USING MACHINE LEARNING
Machine vision Application
Machine Vision On Embedded Platform
Multiple object detection report
Electronic Circuit Assessment using Machine Learning (ML)
Real Time Moving Object Detection for Day-Night Surveillance using AI
What is machine vision slide share
Design, Analysis and Fabrication of Pick & Place Colour Sorting Robotic Arm
IRJET- Analysing Wound Area Measurement using Android App
IRJET- Automated Student’s Attendance Management using Convolutional Neural N...
Human pose detection using machine learning by Grandel
IRJET - Human Pose Detection using Deep Learning
docmentation with rules
Face Detection And Tracking
A SOFTWARE REQUIREMENT ENGINEERING TECHNIQUE USING OOADA-RE AND CSC FOR IOT B...
Internship report on AI , ML & IIOT and project responses full docs
Automated Attendance Management System
Ad

More from Omkar Rane (20)

PDF
Enabling SSL Elasticsearch on server
PDF
Anti lock braking (ABS) Model based Design in MATLAB-Simulink
PPTX
Autosar fundamental
PPTX
Stress Management
PPTX
Bootloaders (U-Boot)
PPTX
Concept of Diversity & Fading (wireless communication)
PPTX
Tata Motors GDC .LTD Internship
PDF
Machine Learning Model for M.S admissions
PDF
Timer 0 programming on LPC 1768
PDF
ADC (Analog to Digital conversion) using LPC 1768
PDF
PWM based motor speed control using LPC 1768
PDF
UART interfacing on LPC1768 (Cortex M3 micro controller)
PDF
LED Blinking logic on LPC1768
PDF
CAN interfacing on LPC1768 (ARM Cortex M3 based Micro controller)
PPTX
Vlisi Course project presentation:Keypad Scanner
PDF
VlSI course project report : Keypad Scanner
PPTX
LPC 1768 A study on Real Time clock features
PDF
Nexys4ddr rm FPGA board Datasheet
PDF
Linear Regression (Machine Learning)
DOCX
transmission gate based design for 2:1 Multiplexer in micro-wind
Enabling SSL Elasticsearch on server
Anti lock braking (ABS) Model based Design in MATLAB-Simulink
Autosar fundamental
Stress Management
Bootloaders (U-Boot)
Concept of Diversity & Fading (wireless communication)
Tata Motors GDC .LTD Internship
Machine Learning Model for M.S admissions
Timer 0 programming on LPC 1768
ADC (Analog to Digital conversion) using LPC 1768
PWM based motor speed control using LPC 1768
UART interfacing on LPC1768 (Cortex M3 micro controller)
LED Blinking logic on LPC1768
CAN interfacing on LPC1768 (ARM Cortex M3 based Micro controller)
Vlisi Course project presentation:Keypad Scanner
VlSI course project report : Keypad Scanner
LPC 1768 A study on Real Time clock features
Nexys4ddr rm FPGA board Datasheet
Linear Regression (Machine Learning)
transmission gate based design for 2:1 Multiplexer in micro-wind

Recently uploaded (20)

PDF
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
DOCX
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
PDF
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
PDF
keyrequirementskkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
PDF
composite construction of structures.pdf
PPTX
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
PDF
Operating System & Kernel Study Guide-1 - converted.pdf
PDF
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
PPT
Mechanical Engineering MATERIALS Selection
PPTX
CYBER-CRIMES AND SECURITY A guide to understanding
PPTX
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
PPTX
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
PPTX
additive manufacturing of ss316l using mig welding
PDF
The CXO Playbook 2025 – Future-Ready Strategies for C-Suite Leaders Cerebrai...
DOCX
573137875-Attendance-Management-System-original
PPTX
Welding lecture in detail for understanding
PPTX
CH1 Production IntroductoryConcepts.pptx
PPT
Project quality management in manufacturing
PPT
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
PPTX
web development for engineering and engineering
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
keyrequirementskkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
composite construction of structures.pdf
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
Operating System & Kernel Study Guide-1 - converted.pdf
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
Mechanical Engineering MATERIALS Selection
CYBER-CRIMES AND SECURITY A guide to understanding
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
additive manufacturing of ss316l using mig welding
The CXO Playbook 2025 – Future-Ready Strategies for C-Suite Leaders Cerebrai...
573137875-Attendance-Management-System-original
Welding lecture in detail for understanding
CH1 Production IntroductoryConcepts.pptx
Project quality management in manufacturing
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
web development for engineering and engineering

Machine Vision On Embedded Platform -Report

  • 1. Minor Project Report on Machine Vision- Application on Embedded Platform Submitted by, Omkar Rane T187014 Abhishek Sainkar T187009 Kaustubh Wankhede T187003 Guided by, Prof. Bhairavi Savant A Report submitted, In partial fulfilment of the requirements for Sixth Semester of BACHELOR OF TECHNOLOGY in ELECTRONICS & TELECOMMUNICATION ENGG. School of Electrical Engineering Alandi (D), Pune – 412105 2018-19
  • 2. School of Electrical Engg. UG Program in E&TC Engg (Accredited by NBA, ISO 9001:2015 Certified) CERTIFICATE This is certify that, Omkar Rane T187014 Abhishek Sainkar T187009 Kaustubh Wankhede T187003 of T.Y. B.Tech. have submitted a Report on, Machine Vision- Application on Embedded Platform The said work is completed by putting the requirement of hours as per prescribed curriculum during the academic year 2018 – 19. The report is submitted in the partial fulfilment of the requirements for the course Mini Project in the Sixth Semester of Degree of Engineering in - Electronics and Telecommunication Engineering of MIT Academy of Engineering. Prof. Bhairavi Savant Dr. Debashis Adhikari Project Advisor Dean, SEE External Examiner
  • 3. Contents Abstract i Acknowledgement ii List of Figures iii List of Tables iv 1. Introduction 1 1.1 Motivation 1 1.2 Objectives and Scope 2 1.3 Problem Statement 3 2. Literature Survey 4 3. System Design 5 3.1 Hardware Requirements 5 3.2 Software Requirements 7 3.3 Methodology 8 4. Implementation Details 16 5. Results 22 6. Conclusion and Future scope 26 Project Plan Bill of Material References
  • 4. i Abstract Our Project mainly focuses on the Machine Vision Application. We have deployed the color segmentation algorithm to detect the objects integrated with embedded board. The Project focuses on image-based object detection technique to recognize the object. We have successfully identified the objects. Our work would be implemented in automatic inspection, process control and robot guidance usually in Industry 4.0
  • 5. ii Acknowledgement We are thankful to all the people who are involved in this project including jury members who suggested rectification of errors in our project from time to time during our presentations. We are also thankful to School of Electrical Engineering who had helped in project. Also, the Guide Prof. Bhairavi Savant for Project by their guidance we were doing the appropriate changes. We are also thankful to Prof. Smita Kulkarni for their guidance We would like to thank to all who knowingly and unknowingly helped us to accomplish this project of third year in sixth semester. Omkar Rane Abhishek Sainkar Kaustubh Wankhede
  • 6. iii
  • 7. iii List of Figures Fig. No. Fig. Name Page No. Fig 3.1 Raspberry Pi 3 Block Diagram 5 Fig 3.2 Raspi Camera 6 Fig 3.3 Block Diagram for Detection 8 Fig 3.4 Raspberry Pi configuration 9 Fig 3.5 Setting upper and lower limit of each colour in HSV. 14 Fig 4.1 HSV Colour Model 18 Fig 4.2 Camera Interfacing with Raspberry Pi 21 Fig 5.1 Compilation of Python file in Shell 22 Fig 5.2 Detection of Red Bottle 22 Fig 5.3 Detection of Yellow Bottle 23 Fig 5.4 Detection of Green Bottle 23 Fig 5.5 Colour detection setup 24 Fig 5.6 Detection of Blue Bottle 24 Fig 5.7 Detection of Light Blue Bottle 25 Fig 5.8 Detection of Dark Blue Bottle 25
  • 8. iv List of Tables Table. No. Table. Name Page No. Table 3.1 Raspberry Pi Specification 6 Table 3.2 PiCamera Specification 7
  • 9. Page | 1 Chapter 1 Introduction 1.1 Motivation For a few decades, computer scientists and engineers have attached cameras and simplistic image interpretation methods to a computer (robot) in order to impart vision to the machine. A lot of interest has been shown towards object recognition, object detection, object categorization etc. Simply speaking, object recognition deals with training the computer to identify a particular object from various perspectives, in various lighting conditions, and with various backgrounds; object detection deals with identifying the presence of various individual objects in an image; and object categorization deals with recognizing objects belonging to various categories. For example, a domestic help robot can be trained to recognize if an object is a coffee machine (object recognition), it may be trained to detect a coffee machine in the kitchen (object detection), and it may be trained to identify cups of various types and forms into a common category called cups.[01] Despite the simplistic definition mentioned above, the lines separating the three skills above are very blur and the problems often intermingle in terms of the challenges as well as solution approaches. Further, it is evident that for practical purposes, a good combination of all the three skills is essential. Great success has been achieved in controlled environment for object detection/recognition problem but the problem remains unsolved in uncontrolled places, in particular, when objects are placed in arbitrary poses in cluttered and occluded environment. As an example, it might be easy to train a domestic help robot to recognize the presence of coffee machine with nothing else in the image. On the other hand, imagine the difficulty of such 2 robot in detecting the machine on a kitchen slab that is cluttered by other utensils, gadgets, tools, etc. The searching or recognition process in such scenario is very difficult. So far, no effective solution has been found for this problem. Despite a lot of research in this area, the methods developed so far are not efficient, require long training time, are not suitable for real time
  • 10. Page | 2 application, and are not scalable to large number of classes. Object detection is relatively simpler if the machine is looking for detecting one particular object (say coffee machine). However, recognizing all the objects inherently requires the skill to differentiate one object from the other, though they may be of same type. Such problem is very difficult for machines, if they do not know about the various possibilities of objects.[02] Besides intra-class and inter-class separation problem, another big problem is how many categories should be considered and what should be the basis for recognizing a category, forming a new category, etc. This also involves the problem of enabling the machine to classify objects in existing categories, if possible, recognizing the objects that do not belong to any existing categories, and learning new categories. For object categorization, there is no exhaustive and exclusive set of objects, which can be used to train the machine.[03] 1.2 Objectives and Scope Scope: For mini project we are restricting ourselves to detection of objects by camera by its color. Object detection is one of major step in machine vision. Given a set of object classes, object detection consists in determining the location and scale of all object instances, if any, that are present in an image. Thus, the objective of an object detector is to find all object instances of one or more given object classes regardless of scale, location, pose, view with respect to the camera, partial occlusions, and illumination conditions. Objective: ➢ Detection of all objects in scope of vision. ➢ Gather Data in form of images. ➢ Train Model for detection of objects. ➢ Segmentation of objects according to user need.
  • 11. Page | 3 1.3 Problem Statement To design and Machine Vision application on embedded platform based on ARM Cortex processor and Embedded Linux operating system and using color segmentation algorithm for detection of objects based upon its color via OpenCV Machine Learning Libraries. Our Aim to detect particular object based upon color combination chosen by user.
  • 12. Page | 4 Chapter 2 Literature Survey 1. O. Alhusain[05] discuss that In the food industry there are problems such as The first one is the decline in food quality, and the second one is the “waste” problem associated with processing and preparation operations. Hence, there is the need for quality inspection and assurance mechanisms to be installed in the production lines of such mass food processing. 2. Jarimopas et al. [06] have described development a machine vision experimentally sorting sweat tamarinds parameter shape, size, colour and defects. It was performed with image processing software analysis the image and hardware design include a conveyor, control drive light source, image antenna, manage unit and micro-computer. 3. Tushar Jain [07] described Machine vision provides innovative solutions in the direction of industrial automation. These activities include, among others, delicate electronics component manufacturing, quality textile production, metal product finishing, glass manufacturing, machine parts, printing products and granite quality inspection, integrated circuits manufacturing and many others. Machine vision technology improves productivity and quality management and provides a competitive advantage to industries that employ this technology. 4. Industries based on automation, consumer markets, medical domains, Défense and surveillance sectors are most likely domains extensively using machine vision. Image classification, being the widely researched area in the domain of computer vision has achieved remarkable results in world-wide competitions.
  • 13. Page | 5 Chapter 3 System Design 3.1 Hardware Requirements: 1) Raspberry Pi 3 b Raspberry Pi single-board computer represent this concept as affordable educational platform. Here at xDevs.com various versions of Raspberry Pi are already widely used as test equipment controllers and live datalogging platform. Low cost (around $40 USD for fully-featured Raspberry Pi 3) and compact size allow easy adding of standalone modules for running different experiments, and invisible to total project budget since involved test equipment often costs many times more.[02] Fig 3.1 Raspberry Pi 3 Block Diagram
  • 14. Page | 6 Heart of the Pi is the Broadcom BCM2837 SOC which has quad-core Cortex-A53 ARM processor and VideoCore IV GPU. As these implementations are proprietary to Broadcom, we don’t know exact details, however for purpose of overclocking it’s less important. GPU supports OpenGL ES standard and capable to decode Full-HD 60fps video content in real-time. Separate 1GB LPDDR2 SDRAM memory chip EDB8132B4PB-8D-F from Elpida (Micron) located on the bottom side. Table 3.1 Raspberry Pi Specification 2. Raspi Camera V1 The Raspberry Pi Camera Board plugs directly into the CSI connector on the Raspberry Pi. It's able to deliver a crystal clear 5MP resolution image, or 1080p HD video recording at 30fps. Fig 3.2 Raspi Camera
  • 15. Page | 7 Table 3.2 PiCamera Specification 3.2 Software Requirements OpenCV: OpenCV (Open Source Computer Vision) is a community driven open source software library, making it a good place to start working with computer vision. An obvious drawback with using this library is the fact that there is no official IDE to use when building applications. To build an application the user is required to use one of the programming languages mentioned in the paragraph below, making this the tool with the steepest learning curve. Due to the immense user base there are however, many forums in which problems and new ideas can be discussed, making the lack of an IDE less of a problem. Written in C++ the library is accessible using C, C++, Python and Java adding to the availability and possibilities when using this library. Furthermore, OpenCV supports most cameras and frame grabbers. The problem that could arise concerning these is: driver compatibility with the computer's operating system. This is however, a possible problem which concerns all three of the libraries in question. The software library is registered as a BSD licence meaning the modification and copying of the source code is allowed
  • 16. Page | 8 without having to share the results with the community. Therefore, it is free to use in commercial, industrial and private projects.[10] OpenCV is written in C++ and its primary interface is in C++, but it still retains a less comprehensive though extensive older C interface. There are bindings in Python, Java and MATLAB/OCTAVE. The API for these interfaces can be found in the online documentation. Wrappers in other languages such as C#, Perl, and Ruby have been developed to encourage adoption by a wider audience. OpenCV runs on the following desktop operating systems: Windows, Linux, macOS, FreeBSD, NetBSD, OpenBSD. OpenCV runs on the following mobile operating systems: Android, iOS, Maemi, Blackberry The user can get official releases from Source Forge or take the latest sources from GitHub. 3.3 Methodology Fig 3.3 Block Diagram for Detection Raspberry Pi Camera Installation: Install the camera module by plugging the camera wire into the port labelled camera, which is next to the HDMI port on the raspberry pi. Once installed update and upgrade as seen below sudo apt-get update sudo apt-get upgrade Next you must enable the camera, this can be done by going into the "Applications Menu", and under "Preferences" select "Raspberry Pi Configuration". Under the "Interfaces" tab press the "Enable" radio button for Camera. Remotely this can be done using "sudo raspi-config", and in "Interfacing Options" enable the camera.[03]
  • 17. Page | 9 Fig 3.4 Raspberry Pi configuration Installation Of OpenCV: OpenCV is an open source computer vision and machine learning software library. OpenCV is released under a BSD license making it free for both academic and commercial use. It has C++, Python, and Java interfaces and supports Windows, Linux, Mac OS, iOS, and Android. OpenCV was designed for computational efficiency and a strong focus on real-time applications. To install OpenCV, we need to have Python installed. Since Raspberry Pis are preloaded with Python, we can install OpenCV directly. Type the commands below to make sure your Raspberry Pi is up to date and to update the installed packages on your Raspberry Pi to the latest versions.[03] sudo apt-get update
  • 18. Page | 10 sudo apt-get upgrade Type the following commands in the terminal to install the required packages for OpenCV on your Raspberry Pi. sudo apt install libatlas3-base libsz2 libharfbuzz0b libtiff5 libjasper1 libilmbase12 libopenexr22 libilmbase12 libgstreamer1.0-0 libavcodec57 libavformat57 libavutil55 libswscale4 libqtgui4 libqt4-test libqtcore4 Type the following command to install OpenCV 3 for Python 3 on your Raspberry Pi, pip3 tells us that OpenCV will get installed for Python 3. sudo pip3 install opencv-contrib-python libwebp6 1. Import Packages First, import the packages required for this project. import cv2 from picamera.array import PiRGBArray from picamera import PiCamera import numpy as np
  • 19. Page | 11 2. Create Your Trackbars Next, we create the trackbars to help us with selecting a color. To create the trackbars, we have the cv2.createTrackbar() function. We have five arguments to create: 1. The trackbar name 2. The window name to which it is attached 3. The default values 4. The maximum value 5. The call back function executed every time trackbar value changes The callback function always has a default argument, which is the trackbar position In our case, the function does nothing, so we will simply pass. We are going to create three trackbars for blue, green, and red. Each trackbar will have a default value of 0 and a maximum value of 255 and will be attached to the window named Trackbars def nothing(x): pass cv2.namedWindow("Trackbars") cv2.createTrackbar("B", "Trackbars", 0, 255, nothing) cv2.createTrackbar("G", "Trackbars", 0, 255, nothing) cv2.createTrackbar("R", "Trackbars", 0, 255, nothing) More
  • 20. Page | 12 3. Setting up the Raspberry Pi Camera Now, we can initialize the camera object that allows us to play with the Raspberry Pi camera. We set the resolution at (640, 480) and the frame rate at 30 fps camera = PiCamera() camera.resolution = (640, 480) camera.framerate = 30 4. The PiRGBArray() PiRGBArray() gives us a 3-dimensional RGB array organized (rows, columns, colors) from an unencoded RGB capture. PiRGBArray gives us the advantage of reading the frames from Raspberry Pi camera as NumPy arrays, making it compatible with the OpenCV. It avoids the conversion from JPEG format to OpenCV format which would slow our process. PiRGBArray()takes 2 arguments: the first is the camera object and the second is the resolution. rawCapture = PiRGBArray(camera, size=(640, 480) 5. The capture continuous Function Now we are going to use the capture_continuous function to start reading the frames from the Raspberry Pi camera module.The capture_continuous function takes three arguments: 1) rawCapture
  • 21. Page | 13 2) The format in which we want to read each frame. Since OpenCV expects the image to be in the BGR format rather than the RGB, we need to specify the format to be BGR. 3) The use_video_port boolean. Making this true means that we treat the stream as video. Once we have the frame, we can access the raw NumPy array via the .arrayattribute. for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True): image = frame.array 6. Setting up the Color Recognition Now we are going to convert images from the BGR to the HSV color space. The HSV (hue saturation value) space gives us better results when doing color-based segmentation. hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV Next, we create the trackbars used to select the color. B = cv2.getTrackbarPos("B", "Trackbars") G = cv2.getTrackbarPos("G", "Trackbars") R = cv2.getTrackbarPos("R", "Trackbars") After that, we can find out the lower and upper limit of the color in HSV. green = np.uint8([[[B, G, R]]]) hsvGreen = cv2.cvtColor(green,cv2.COLOR_BGR2HSV)
  • 22. Page | 14 lowerLimit = np.uint8([hsvGreen[0][0][0]-10,100,100]) upperLimit = np.uint8([hsvGreen[0][0][0]+10,255,255]) Fig 3.5 Setting upper and lower limit of each colour in HSV. Next, we adjust the threshold of the HSV image for a range of each selected color. mask = cv2.inRange(hsv, lowerLimit, upperLimit) Now we can extract the objects of the colors in the frame. result = cv2.bitwise_and(image , image , mask=mask) Let’s show the result in the output window. cv2.waitkey() is a keyboard binding function and waits for a specified amount of time for any keyboard event. It takes only one argument, which is set to time in milliseconds. If the key is pressed in that time then the program will continue. Passing 0 means it will wait indefinitely for a key. cv2.imshow("frame", image)
  • 23. Page | 15 cv2.imshow("mask", mask) cv2.imshow("result", result) key = cv2.waitKey(1) Always clear the stream in preparation for the next frame by calling truncate (0) between captures. rawCapture.truncate(0)
  • 24. Page | 16 Chapter 4 Implementation Details 1. Image Segmentation The process of partitioning a digital image into multiple segments is defined as image segmentation. Segmentation aims to divide an image into regions that can be more representative and easier to analyze. Such regions may correspond to individual surfaces, objects, or natural parts of objects. Typically, image segmentation is the process used to locate objects and boundaries (e.g., lines or curves) in images. Furthermore, it can be defined as the process of labeling every pixel in an image, where all pixels having the same label share certain visual characteristics. Usually segmentation uses local information in the digital image to compute the best segmentation, such as color information used to create histograms or information indicating edges, boundaries, or texture information [10]. Color image segmentation that is based on the color feature of image pixels assumes that homogeneous colors in the image correspond to separate clusters and hence meaningful objects in the image. In other words, each cluster defines a class of pixels that share similar color properties. As the segmentation results depend on the used color space, there is no single-color space that can provide acceptable results for all kinds of images. For this reason, many authors tried to determine the color space that will suit their specific color image segmentation problem. In this work, a segmentation of color images is tested with different classical color spaces, RGB, CMY, XYZ & HSV to select the best color space for the considered kind of images.[10] The segmentation process is based on the Grab Cut segmentation technique, which is considered as one of the powerful state-of-the-art techniques for the problem of color image segmentation. The iterative energy minimization scheme of the Grab Cut is based on the powerful optimization of the Graph Cut technique which allows for the generation of the global optimal segmentation. In addition, Graph Cut can be easily well extended to the problem of N-D images. Furthermore, the cost energy function of the Graph Cut
  • 25. Page | 17 minimization process allows it to be defined in terms of different image features such as color, region, boundary, or any mixture of image features. This flexibility provides wide potential for the use of Grab Cut in different applications. On the other hand, GrabCut is considered as a billable segmentation technique, where images can be segmented into two background and foreground regions only. Initial user intervention is required in order to specify an object of interest to be segmented out of the image, considering all the remaining image pixels as one background region. This classifies the GrabCut as a semiautomatic segmentation technique and turns the quality of the initialization and hence the segmentation performance, sensitive to the user selection. In other words, poor GrabCut initialization may lead to bad final segmentation accuracy which might require extra user interactions with the segmentation results for fine tuning How does Image Segmentation work? We can divide or partition the image into various parts called segments. It’s not a great idea to process the entire image at the same time as there will be regions in the image which do not contain any information. By dividing the image into segments, we can make use of the important segments for processing the image. That, in a nutshell, is how image segmentation works. An image is a collection or set of different pixels. We group together the pixels that have similar attributes using image segmentation. Take a moment to go through the below visual (it’ll give you a practical idea of image segmentation.[10] Object detection builds a bounding box corresponding to each class in the image. But it tells us nothing about the shape of the object. We only get the set of bounding box coordinates. We want to get more information – this is too vague for our purposes. Image segmentation creates a pixel-wise mask for each object in the image. This technique gives us a far more granular understanding of the object(s) in the image. Why do we need to go this deep? Can’t all image processing tasks be solved using simple bounding box coordinates? Let’s take a real-world example to answer this pertinent question.
  • 26. Page | 18 2. The hue-saturation-value (HSV) color model Another way to characterize a color is in terms of the HSV model. •The hue (H) of a color refers to which pure color it resembles. All tints, tones and shades of red have the same hue. Hues are described by a number that specifies the position of the corresponding pure color on the color wheel, as a fraction between 0 and 1. Value 0 refers to red; 1/6 is yellow; 1/3 is green; and so forth around the color wheel. •The saturation (S) of a color describes how white the color is. A pure red is fully saturated, with a saturation of 1; tints of red have saturations less than 1; and white has a saturation of 0. •The value (V) of a color, also called its lightness, describes how dark the color is. A value of 0 is black, with increasing lightness moving away from black. This diagram, called the single-hex cone model of color space, can help you visualize the meaning of the H, S, and V parameters. Fig 4.1 HSV Colour Model
  • 27. Page | 19 •The outer edge of the top of the cone is the color wheel, with all the pure colors. The H parameter describes the angle around the wheel. •The S (saturation) is zero for any color on the axis of the cone; the center of the top circle is white. An increase in the value of S corresponds to a movement away from the axis. •The V (value or lightness) is zero for black. An increase in the value of V corresponds to a movement away from black and toward the top of the cone. The Ostwald diagram corresponds to a slice of this cone. For example, the triangle between red, white, and black is the Ostwald diagram for the varieties of red. 3. RGB to HSV conversion formula The R, G, B values are divided by 255 to change the range from 0..255 to 0..1: R' = R/255 G' = G/255 B' = B/255 Cmax = max(R', G', B') Cmin = min(R', G', B') Δ = Cmax – Cmin Hue calculation:
  • 28. Page | 20 Saturation calculation: Value calculation: V = Cmax 4. Algorithm Step 1: Run shell Command python proj.py Step 2: Import OpenCV2 library as CV2 Step 3: Import Numpy Library as np Step 4: from picamera.array import PiRGBArray Step 5: from picamera import PiCamera Step 6: Create trackbar cv2.Trackbar() for R,G,B for colour Segmentation values. Step 7: Set Resolution of camera and frame rate for image specification. camera.resolution = (640, 480) camera.framerate = 30 Step 8: Capture Continuous Frames in Camera for Image Acquisition. Step 9: Do Background Subtraction. Step 10: Convert Red, Green and Blue to Hue Saturation Value. Step 11: get lower and upper limit value from HSV values. Step 12: Mask the values received from lower and upper limit of HSV. Step 13: Close window.
  • 29. Page | 21 5. Raspberry Pi with Camera Fig 4.2 Camera Interfacing with Raspberry Pi
  • 30. Page | 22 Chapter 5 Results Fig 5.1 Compilation of Python file in Shell Fig 5.2 Detection of Red Bottle
  • 31. Page | 23 Fig 5.4 Detection of Green Bottle Fig 5.3 Detection of Yellow Bottle
  • 32. Page | 24 Fig 5.5 Colour detection setup Fig 5.6 Detection of Blue Bottle
  • 33. Page | 25 Fig 5.8 Detection of Dark Blue Bottle Fig 5.7 Detection of light Blue Bottle
  • 34. Page | 26 Chapter 6 Conclusion and Future Scope In this project, an attempt was made to develop an object detection and tracking framework able to run in real-time on a Raspberry Pi 3 Model B. By using colour segmentation method, we were able to classify objects. This Project has future scope for implementation such detection for industrial Automation and quality control practices. OpenCV as the vision library of choice, is more than large and powerful enough to build a program of this type. The documentation supporting the library is well written and easy to understand. The wealth of examples and explanations in the OpenCV community have been able to answer any and all questions that arose during the development process, and has been invaluable as a source of knowledge. The program has been developed, exclusively, using open source software. This points to the fact that further development would most likely lead to a stable and cheap solution that could be incorporated into a commercial product, with no attached licensing fees. The argument can also be made, concerning the hardware, that the Raspberry Pi, while being a good platform during development, is too costly as the ultimate hardware in production. The hardware performance can definitely be lower, than what the Raspberry Pi supplies, and still perform at a satisfying speed. Thus, cutting the cost of a final product even further. In the future, we would make Real time object detection which will be able to distinguish them, also count the number of objects in front.
  • 35. PROJECT PLAN Sr. No Week Work to Be Done Remark 1 1st Deciding of Topic Completed 2 2nd Literature Survey Completed 3 3rd Installing OS on Raspberry Pi Completed 4 4th & 5th Raspberry Pi Configuration Completed 5 6th Purchased Camera Completed 6 7th Configure camera with raspberry Pi Completed 7 8th & 9th Install OpenCV on Raspberry Pi Completed 8 10th Installed Supportive libraries of OpenCV Completed 9 11th & 12th Python Coding Completed 10 13th Image Segmentation Completed 11 14th & 15th Testing Completed 12 16th Result Collection Completed 13 17th & 18th Report Writing Completed
  • 36. BILL OF MATERIAL Sr no Component Price (Rs) 1 Raspberry Pi 3 Model B 3500 2 PICAMERA REES52 800 Total: Rs. 4300
  • 37. REFERENCES [ 01 ] Bradski, G., Kaehler A. Learning OpenCV: Computer Vision in C++ with the OpenCV Library. Sebastopol, California : O’Reilly Media, Inc., 2008. [ 02 ] http://eie.uonbi.ac.ke/sites/default/files/cae/engineering/eie/Computer%20Vision%20thro ugh %20the%20Raspberry%20PI%20Counting%20Objects.pdf [ 03 ] https://www.pyimagesearch.com/2018/09/26/install-opencv-4-on-your-raspberry-pi/ [ 04 ] https://medium.com/nanonets/how-to-easily-detect-objects-with-deep-learning-on- raspberrypi-225f29635c74 [ 05 ] O. Alhusain, VISION-BASED SYSTEM FOR QUALITY CONTROL OF SOME FOOD PRODUCTS. Wilson.,2003 [ 06 ] Jarimopas, Nitipong Jaisin, An experimental machine vision system for sorting sweet tamarind.. 2008, Elsevier [ 07 ] Tushar Jain, Meenu. Automation and Integration of Industries through Computer Vision Systems..2013, IRPHOUSE [ 08 ] What is a Raspberry Pi?”. Raspberry Foundation, 2015. [Online]. Available: https://www.raspberrypi.org/help/what-is-a-raspberry-pi/. [ 09 ] https://www.rs-online.com/designspark/object-tracking-using-computer-vision-and- raspberry-pi [ 10 ] https://www.hindawi.com/journals/tswj/2014/126025/