Multisensor data fusion based autonomous mobile

IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308
_______________________________________________________________________________________
Volume: 03 Special Issue: 01 | NC-WiCOMET-2014 | Mar-2014, Available @ http://www.ijret.org 75
MULTISENSOR DATA FUSION BASED AUTONOMOUS MOBILE
ROBOT WITH MANIPULATOR FOR TARGET DETECTION
Chinchu Chandrasenan1
, Nafeesa T.A2
, Reshma Rajan3
, Vijayakumar K4
1, 2, 3
P.G Student, 4
Professor, Electronics & Communication Engineering, Toc-H Institute of Science & Technology,
Kerala, India
Abstract
This paper proposes a novel autonomous mobile robot in unknown environment navigating through obstacles by computing the
shortest path. Flood fill algorithm is used for path planning of the mobile robot. The concept of minimum energy contour to perform
the desired operation of robotic manipulator is achieved using Fuzzy algorithm. Object detection is done by utilizing a mobile robot
with sensors and object recognition is achieved by image processing using Principal Component Analysis (PCA). This paper
describes the implementation of multi sensor data fusion assisting a mobile robot to acquire a purposive behavior in the respective
environment. This is achieved by directly integrating sensor information which helps the robot to successfully navigate and also
enables fetch & retrieval operation of robotic manipulator. In this approach, information is taken from distance sensors, position
sensors and image sensor. This work also aims to provide an optimal fusion of information from distributed multiple sensors using
Kalman Filter.
Keywords: Mobile robot navigation, flood fill, robotic manipulator, fuzzy algorithm, object detection, multi-sensor fusion,
Principal Component Analysis, Kalman Filter.
----------------------------------------------------------------------***------------------------------------------------------------------------
1. INTRODUCTION
By autonomous robot navigation we mean the ability of a
robot to move purposefully and without human intervention in
environments that have not been specifically engineered for it.
Autonomous robot is equivalent to a closed-loop control
system in which the robot is the object to be controlled, the
decision-making system is the system controller and the vision
system performs the functions of detection and feedback
loops. The visual system is mainly to capture, process and
recognize images [7]. When the autonomous robot moves, it
relies on artificial intelligence and target vision acquisition.
Important advances have been made in the last period in the
robotic domain. The navigation of the mobile robots requires:
the current position, the path planning and the obstacle
avoidance. The environment around the robot is not known, so
it must have decision-making capabilities [33]. Path planning
is an important part of autonomous mobile robot, and
according to some evaluation standards, it finds a collision-
free path from original state to target state in obstacle
environment [1]. The flood-fill algorithm is used primarily for
path planning. It is used to plan an optimal (shortest) path to
the nearest unexplored cell in the event that a repeated state is
detected and it is also used to plan a path back "home" when
the goal is reached. Fuzzy algorithm is implemented for the
motion plan of robotic manipulator. The robotic manipulator
adapts the shortest path to reach the target and follow a
minimum energy curve to perform the desired function. Fuzzy
algorithm provides solutions in a sufficiently short amount of
time with minimum energy consumption.
In most of the mobile robotics navigation scenarios , for the
robot to operate in an unknown dynamic environment, it is
necessary to integrate or to fuse the data from different types
of sensors so as to obtain useful information from the
respective environment. The main advantage of using multi-
sensor systems is the increase in reliability and flexibility
provided by the redundant and diverse sensor information
[31]. Here, multisensory data is fused using Kalman filter
which enables a mobile robot to accomplish a given task by
directly coupling multi sensor information and actions through
interaction between the robot and its environment.

_______________________________________________________________________________________
When the autonomous mobile robot moves, it depends on
vision system to catch the targeted objects. So vision system is
one of the key technologies for intelligent robot. Principal
Component Analysis has been investigated for appearance-
based object recognition[24]. This method has been found
attractive as it compresses the data.
2. ALGORITHM
2.1 Path Planning
Autonomous robots which work without any human
intervention are required in robotic fields. The robot has to
move in any environment even in the one it has never seen
before. This robot is essentially designed to move on a floor.
When a robot moves in the given environment from starting
point to the target point it is necessary to plan an optimal or
feasible path. It must avoid obstacles coming on its way. In
this paper the well known Flood fill algorithm is implemented
to make the mobile robot navigate. After reaching the target
position, it finds the shortest path from target position to an
initial position.
The entire unknown environment is divided into grids with
static obstacles. The robot moves within the unknown
environment by sensing and avoiding the obstacles coming
across its way towards the target. When the mission is
executed, it is necessary to plan an optimal or feasible path for
itself avoiding obstructions in its way and minimizing a cost
function such as time, energy, or distance [1].
Flood fill algorithm is one of the most efficient maze solving
algorithms. The flood fill algorithm is derived from the
“Bellman Ford Algorithm”. Using this method complex and
difficult mazes can be solved efficiently. The algorithm works
by assigning values for all cells in the maze, where these
values indicate the steps from any cell to the source cell. It is a
very efficient method to solve even a complicated maze. Here
flood fill algorithm has been used to drive the robot to solve a
real environment as in our case. This robot uses various
sensors that help in navigation.
The algorithm consists of two phases. In the initial phase, the
unknown maze like environment is divided into equal sized
grids and the cells are flooded with default value of -1
initially. The values in the cell changes as the robot start
moving. The flooded weighted value of a cell represents the
number of steps that would be required to reach that particular
cell from the initial cell. As the robot starts exploring the maze
and obstacles in each cell, it has to update the weighted value
because the number of steps from source cell to reach that
particular cell would change. The shortest path will be in the
decrementing order of the grid values which is the shortest
path.

_______________________________________________________________________________________
Proposed pseudo code for executing the process of updating
the weighted value of each cell.
a) Start scanning from the start node(0,0)
b) Scan in determined pattern
c) Is sensors detected obstacle? do step d or e
d) No →go to step h
e) Yes → turn and move the robot for the next available
position and go to step h.
f) Object detected? and do step f or g
g) Yes → Stop
h) No →go to step h
i) Change the cell to the value of the cell +1, if the robot
moves in up or right direction, and Change the cell to the
value of the cell -1 ,if the robot moves in down or left
direction, go to step b
2.2 Robotic Manipulator
Robotic manipulators are becoming multifunction
programmable manipulation devices designed to do different
tasks in various fields. An efficient algorithm based robotic
manipulator is necessary to increase the accuracy in various
fields. The well known fuzzy algorithm is implemented for the
motion plan of robotic manipulator[5]. The robotic
manipulator finds the shortest path to reach the target and
follows a minimum energy contour to perform the desired
function. Path planning of robotic manipulator based on fuzzy
algorithm provide solutions in a sufficiently short amount of
time with minimum energy consumption
Robotic manipulator uses sensors to determine position of
object within its working envelop. Visual sensor is used to
identify shape, position and orientation of object. The
controller accepts the sensor data to acquire the desired
position of robotic manipulator. It will also ensure that the
robotic manipulator is correctly positioned to initiate the
object acquisition. Controller provides the necessary signals
for controlling the manipulator motors. Manipulator feedback
sensors ensure smooth manipulator and grip operation without
any collision or miss hit.
1) Path planning of Robotic Manipulator: Fuzzy algorithm
provides solutions in a sufficiently short amount of time with
minimum energy consumption. [35]The entire workspace
scanned by camera can be utilized. If target is detected, fuzzy
units are fed with the x and y coordinates of target within
workspace. If no target is detected, the fuzzy unit is informed
that the target is far away. The output variable of each unit is
the motor command .The motor command is given to the link
motor which is fed to the manipulator at each iteration.
2) Fuzzification: Fuzzification module performs two tasks.
Input normalization, mapping of input values into
normalized universe of discourse. Transformation of crisp
process state values into fuzzy sets, in order to make them
compatible with antecedent parts of linguistic rules that will be
applied in fuzzy interface engine [5]. Before path planning
work space of robotic manipulator is divided equally
Then fuzzification of distance in x coordinate and y
coordinate done.
a. Fuzzification of y coordinate
Manipulator can detect an object within a certain range of
distance in Y axis from 0-30 cm. Membership function for
distance can be expressed in cm.
3 fuzzy sets:-
L-LOW
M- MEDIUM
H-HIGH
b. Fuzzification of x coordinate
Manipulator can detect an object within a certain range of
distance in x axis from 0-30 cm. The membership function
for distance can be expressed by 3 fuzzy sets:-
SL = SMALL LEFT
LL = LARGE LEFT
C = CENTRE
SR = SMALL RIGHT
LR = LARGE RIGHT
3) Fuzzy Interference Rules: In a fuzzy interface system, a
rule base is constructed to control the output variable. A fuzzy
rule is a simple IF-THEN rule with a condition and a
conclusion.
4) Fuzzy Rule Table
Fig-2: Fuzzy rule table
Rules can be constructed as follows
If X distance is LL and Y distance is H then control
action is LLH,
If X distance is LL and Y distance is M then control
action is LLM
5) Defuzzification: The last step in the fuzzy inference
process is defuzzification. Fuzzification helps us to evaluate
the rules, but the final output of a fuzzy system has to be a
crisp number. The input for the defuzzification process is the

_______________________________________________________________________________________
aggregate output fuzzy set and the output is a single number.
There are several defuzzification methods, but probably the
most popular one is the centroid technique.
( )
( )∫
∫
= b
a
A
b
a
A
dxx
xdxx
COG
µ
µ
2.3 Sensors
When designing a system using multiple sensors, it is
important to understand the advantages and limitations of each
of the sensors such as tri- axis accelerometer, tri-axis
magnetometer, infrared sensor, ultrasonic sensor, visual
sensor[32]. An accelerometer measures acceleration (change
in speed) of the moving platform. Accelerometers are very
important in the sensor world because they can sense such a
wide range of motion. They are Analog Sensors capable of
measuring acceleration on three axes (up/down, left/right, and
forward/ backward) at the same time. This data could allow
the robot to calculate its velocity or react to collisions and
avoid hit.
The magnetometer senses the earth magnetic field (0.5Gauss–
0.6Gauss). In static conditions, the projection of geomagnetic
field on the three axes allows to compute heading angle.An
algorithm is proposed to detect orientation in three
dimensions. The inertial measurement unit (IMU) is composed
of a tri-axis accelerometer, and a tri-axis magnetometer. A
Kalman filter is implemented to yield the reliable orientation.
The raw data from each sensor need to be calibrated. To
calibrate these data, scale and bias must be taken into account.
The bias represents how far the centre of data is from the zero.
The scale means how much larger the range of data from the
sensor is than the real meaningful data.
Proximity/distance sensors seem to be quite appealing for their
acceptable cost-to performance ratio, as compared to that of
more expensive sensing techniques, e.g., vision or laser range
finding. Among proximity/distance sensors, ultrasonic (US)
and infrared (IR) detectors are particularly interesting in real-
life applications. Our Robot is equipped with simple ultrasonic
sonar for detecting obstacles and an active infrared
communication and localization system. Also our approach
aims to construct an autonomous agent in which both the
functions “perception for action” and “action for perception”
emerges simultaneously by means of the integration of
distance and visual sensor data to accomplish the task of
navigation and pick & place by manipulator. Distance sensor
is used to avoid obstacles and a visual sensor is used to
recognize what the object is.
2.4 Kalman Filter
Fig-3: Block for sensor data fusion
Kalman filter has traditionally been used extensively in the
solution of tracking, estimation and signal extraction
problems. Kalman filtering is an optimal recursive data
processing algorithm that is based upon state space concepts.
The recursive nature of the algorithm makes it suitable for
systems without large data storage capacities.
The Kalman filter [15] is used in sensor fusion and data
fusion. Typically real time systems produce multiple
sequential measurements rather than making a single
measurement to obtain the state of the system. These multiple
measurements are then combined mathematically to generate
the system's state at that time instant.
The following steps are implemented [15] in discrete Kalman
filter:
1) State prediction:
2) Prediction of error covariance:
3) Calculate the constant gain and update
4) Update error covariance:
In this case, the input value to the Kalman filter is (state)
corresponding to the distance between the camera and the
object. In (1) and (2) the value and covariance is predicted
to the next step. In (3), (4) and (5) the equations correct the
discrete Kalman filter. In (3), a new gain of Kalman is
calculated. (4) and (5) calculate a new value of predicted,
and new covariance of error, respectively.
2.5. Object Detection
In this paper, PCA is applied for object detection. Principal
component analysis (PCA) is a typical approach in pattern
recognition. PCA is a way of identifying patterns in data.
Since patterns in data can be hard to find in data of high
dimension, where the luxury of graphical representation is not
(1)

_______________________________________________________________________________________
available, PCA is a powerful tool for analyzing data. The other
main advantage of PCA is that once you have found these
patterns in the data, and you compress the data, ie. by reducing
the number of dimensions, without much loss of information.
Edge detection and color detection is in the forefront of image
processing for object detection. The main steps include
• Get Image data
• Subtract the mean from each vector to get a set of
vectors
• Calculate the Covariance Matrix
• Find the normalized Eigenvectors and Eigen values
of
• Arrange the Eigenvectors according to Eigen values
from highest to lowest
• Find the Feature vector F using the transpose of
Eigenvectors.
There are 2 phases for object detection.
In Training phase, the image of the target is fed to the
PC. Features are extracted and PCA is done.
In detection phase, feature extraction and PCA is
done and matched with already available result to
detect object.
3. IMPLEMENTATION
Figure shows the entire system including the sensors, camera
and robotic platform.
Fig-4: Robot Model
4. CONCLUSIONS
The mobile robot under development has robustness of
conventional path planning techniques. This is by using Flood
fill algorithm for path planning and Principal Component
Analysis for object detection. An efficient target acquisition
and faster mission completion can be achieved by sensor data
fusion through Kalman filtering. Better performance of
robotic manipulator can be achieved through fuzzy path
planning algorithm. Fuzzy algorithm provide solutions in a
sufficiently short amount of time with minimum energy
consumption
ACKNOWLEDGMENTS
The authors wish to thank Dr. Suryakala C.D and Sudheesh.
M for valuable discussions.
REFERENCES
[1] A.Francy Golda,S.Aridha,”Algorithmic Agent for
Effective Mobile Robot Navigation in an Unknown
Environment” ,2009 IEEE
[2] Adam N.Bingaman, ‘Tilt-Compensated Magnetic Field
Sensor’, master dissertation, Virginia Polytechnic
Institut and State University, May 2010
[3] Albert M. Cook, Brenda Bentz, Norma Harbottle,
Cheryl Lynch, and Brad Miller.“School-Based Use of a
Robotic Manipulator System by Children With
Disabilities”.IEEE transactions on neural systems and
rehabilitation engineering, vol. 13, no. 4, december
2005.
[4] Amalia F.Foka,”Sobel Edge Detection Algorithm”
,International Journal of Computer Science and
Management Research Vol 2 Issue 2 February 2013
[5] Amitava Chatterjee, Ranajit Chatterjee, ,Fumitoshi
Matsuno, and Takahiro Endo, “Augmented Stable
Fuzzy Control for Flexible Robotic Manipulator Using
LMI Approach and Neuro-Fuzzy State Space
Modeling”. IEEE transactions on industrial electronics,
march 2008 .
[6] Asim M. Murshid, Sajad A. Loan, Shuja A. Abbasi,
and Abdul Rahman M. Alamoud .“VLSI Architecture of
Fuzzy Logic Hardware Implementation”, International
Journal of Fuzzy Systems, June 2011
[7] Atul Patel, Anupam Dubey, Ajitesh Pandey, Siddharth
Dutt Choubey” Vision Guided Shortest Path Estimation
Using Floodfill Algorithm for Mobile Robot
Applications” 2nd International Conference on Power,
Control and Embedded Systems,2012
[8] Dr. Arti Khaparde , Sowmya Reddy.Y Swetha
Ravipudi , “Face Detection Using Color Based
Segmentation and Morphological Processing””
[9] Duncan Smith And Sameer Singh, Member, IEEE,
“Approaches To Multisensor Data Fusion In target
Tracking: A Survey Duncan Smith”, IEEE
Transactions On Knowledge And Data Engineering,
Vol. 18, No. 12, December 2006.
[10] Edward Cheung , “Proximity Sensing in Robot
.Manipulator Motion Planning: System and
Implementation Issues”, IEEE transactions on robotics
and automation,December l989
[11] Fei-yue wang, “Agent-Based Control for Fuzzy
BehaviorProgramming in Robotic Excavation”, IEEE
transactions on fuzzy systems, august 2004

_______________________________________________________________________________________
[12] Fernando Lizarralde Eduardo V.L Nunes, Liu Hsu,
John T. Wen, IEEE, “Mobile Robot Navigation Using
Sensor Fusion”, IEEE Int. Conf. On Robotics And
Automation, 2003.Filter”, Design 7, no. 1 (2001): 1-16
[13] G.T. Shrivakshan, Dr.C. Chandrasekar” A Comparison
of various Edge Detection Techniques used in Image
Processing” IJCSI International Journal of Computer
Science Issues, Vol. 9, Issue 5, No 1, September 2012
[14] Gian Luca Foresti, Senior Member, Ieee, And Carlo S.
Regazzoni, Senior Member, IEEE, “Multisensor Data
Fusion For Autonomous Vehicle Navigation In Risky
Environments”, IEEE Transactions On Vehicular
Technology ,Vol.51, No.5,september 2002.
[15] Greg Welch and Gary Bishop, “An Introduction to the
Kalman Filter”, Design 7, no. 1 (2001): 1-16.
[16] He Zhao and Zheyao Wang, Member, IEEE, “Motion
Measurement Using Inertial Sensors, Ultrasonic
Sensors, and Magnetometers With Extended Kalman
Filter for Data Fusion”, IEEE SENSORS JOURNAL,
VOL. 12, NO. 5, MAY 2012
[17] Himanshu Borse, Amol Dumbare, Rohit Gaikwad &
Nikhil Lende” Mobile Robot for Object Detection
Using Image Processing” Global Journal of Computer
Science and Technology Neural & Artificial
Intelligence Volume 12 Issue 11 Version 1.0 Year 2012
[18] In So Kweon And Takeo Kande, IEEE, “High
Resolution Terrain Map From Multiple Sensor Data”,
IEEE Transactions On pattern Analysis And Machine
Intelligence ,Vol.14,no.2,february 1992.
[19] Inad A,Ahmed, “Object Recognition System using
Template Matching Based on Signature and Principal
Component Analysis” ,International Journal of Digital
Information and Wireless Communications (IJDIWC)
2(2): 156-163 The Society of Digital Information and
Wireless Communications, 2012 (ISSN 2225-658X)
[20] Inad A. Aljarrah ,Ahmed S. Ghorab ,Ismail M. Khater”
Object Recognition System using Template Matching
Based on Signature and Principal Component
Analysis” International Journal of Digital Information
and Wireless Communications (IJDIWC) 2(2): 156-163
2012
[21] Jiang Dong, Dafang Zhuang, Yaohuan Huang And
Jingying Fu , “Advances In Multi- Sensor Data Fusion
: Algorithms And Applications”, 2009.
[22] JIANG Xin-song, Robotics Introduction. Liaoning
technological publishingfirm,1994
[23] Kai-tai Song And Charles C. Chang, IEEE, “Reactive
Navigation In Dynamic Environment Using
Multisensor Predictor”, IEEE Transactions On
Systems, Man, Cybernetics,vol.29, No.6,
December,1999.
[24] Kui Liu,He Yang,”A Joint Optical Flow and Principal
Component Analysis Approach for Motion Detection” ,
pge 1178-1181,2010 IEEE
[25] M.J. Caruso, Application o magnetoresistive Sensors in
Navigation Systems, Sensors and Actuators, SAE SP-
1220, Feb. 1997, pp. 15-21
[26] Manoj Vairalkar “Edge Detection of Images Using
Sobel Operator” ,International Journal of Emerging
Technology and Advanced Engineering Website:
www.ijetae.com (ISSN 2250-2459, Volume 2, Issue 1,
January 2012)
[27] Mr. Manoj K.Vairalkar , Prof. S.U.Nimbhorkar“ Edge
Detection of Images Using Sobel Operator”
.International Journal of Emerging Technology and
Advanced Engineering .
[28] Peter Kuscera, “Sensors For Mobile Robot Systems”,
Amanipulators, Vol.5, No.4, 2006.
[29] Peter Martin, “Real-time Neuro-Fuzzy Trajectory
Generation for Robotic
[30] Quoc V. Le and Andrew Y. Ng “Joint calibration of
multiple sensors”.
[31] R. C. Luo and G. K. Kay “Multisensor Integration and
Fusion in Intelligent Systems”. IEEE Trans. on
Systems, Man and Cybernetics,19(5):901–931, 1989
Rehabilitation Therapy”, 2009.
[32] Remo Pillat and Arjun Nagendran .“Compliance
Estimation during Bilateral Teleoperation of a Robotic
Manipulator”,2012 IEEE
[33] Ren C. Luo, Fellow, Chih Chia Chang, And Chun Chi
Lai, IEEE, “Multisensor Fusion And Integration:
Theories, applications, And Its Perspectives”, IEEE
Sensors Journal, Vol. 11, No. 12, December 2011.
[34] Sajjad einy,Anoop M. Namboodiri” Object detecting
using PCA image reconstruction and Optical flow”
International Journal of Engineering Research &
Technology (IJERT) Vol. 1 Issue 5, July - 2012
[35] Sandeep Yadav, Kamal Kumar “The Maze problem
solved by Micro mouse” ,International Journal Of
Engineering and Technology April 2012 Siti khairani
binti alias, “Automatic tracking object using simple
robotic manipulator” , OCTOBER 2010
[36] Tascillo, A. Bourbakis, N. “Neural and fuzzy robotic
hand control”, Oct 1999V. Chapnik, G. R. Heppler and
J. D. Aplevich , “Controlling the Impact Response of a
One-Link Flexible Robotic Manipulator”. IEEE
transactions on robotics and automation, vol. 9, no. 3,
june 1993.
BIOGRAPHIES
Chinchu Chandrasenan received the B.Tech.
degree in Electronics and Communication
Engineering from the Cochin University,
India, currently doing M.Tech in VLSI and
Embedded Systems in Cochin University.

_______________________________________________________________________________________
Nafeesa T A received the B.Tech. degree in
Electronics and Communication Engineering
from the Cochin University, India, currently
doing M.Tech in VLSI and Embedded
Systems in Cochin University.
Reshma Rajan received the B.Tech. degree in
Electronics and Communication Engineering
from Vinayaka Mission University, India,
currently doing M.Tech in VLSI and
Embedded Systems in Cusat University.
Prof. K.VijayaKumar received his M.E in
Electronics and Communication from Indian
Institute of Science, Bangalore. He has 37
years of experience in Defence Research &
Development. Presently professor in the
Department of ECE, TocH Institute of Science and
Technology, India. His main field of interests are Signal
Processing and Robotics.

Multisensor data fusion based autonomous mobile

More Related Content

What's hot (11)

Viewers also liked (20)

Similar to Multisensor data fusion based autonomous mobile (20)

More from eSAT Publishing House (20)

Recently uploaded (20)

Multisensor data fusion based autonomous mobile