SlideShare a Scribd company logo
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
DOI : 10.5121/ijma.2014.6602 21
LEADER FOLLOWER FORMATION CONTROL
OF GROUND VEHICLES USING CAMSHIFT
BASED GUIDANCE
S.M.Vaitheeswaran, Bharath.M.K, and Gokul.M
Aerospace Electronics and Systems Division, CSIR-National Aerospace laboratories,
HAL Airport Road, Kodihalli, Bengaluru, 560017 India
ABSTRACT
Autonomous ground vehicles have been designed for the purpose of that relies on ranging and bearing
information received from forward looking camera on the Formation control . A visual guidance control
algorithm is designed where real time image processing is used to provide feedback signals. The vision
subsystem and control subsystem work in parallel to accomplish formation control. A proportional
navigation and line of sight guidance laws are used to estimate the range and bearing information from the
leader vehicle using the vision subsystem. The algorithms for vision detection and localization used here
are similar to approaches for many computer vision tasks such as face tracking and detection that are
based color-and texture based features, and non-parametric Continuously Adaptive Mean-shift algorithms
to keep track of the leader. This is being proposed for the first time in the leader follower framework. The
algorithms are simple but effective for real time and provide an alternate approach to traditional based
approaches like the Viola Jones algorithm. Further to stabilize the follower to the leader trajectory, the
sliding mode controller is used to dynamically track the leader. The performance of the results is
demonstrated in simulation and in practical experiments.
KEYWORDS
Continuously Adapted Mean Shift, Local Binary Pattern (LBP), Kalman filtering, leader-follower,
formation control.
1. INTRODUCTION
The problem of controlling a formation of ground vehicles of autonomous ground vehicles
continues to draw the attention of researches, thanks to its applications in complex tasks like
search and rescue [1], mapping of unknown environments [2], and mobile robotics [3].By
formation control we simply mean the problem of controlling the relative position and orientation
of the vehicles in a group while allowing the group to move as a whole [4]. The use of formation
control enables one to achieve complex tasks with a low robot complexity since each particular
task can be specialized in one task of manipulation or robot navigation. Perception, path
generation and control characteristics, generally decide the formation control process for their
robustness. Also the system architectures provide the infrastructure upon which formation control
is implemented and determines the capabilities and limitations of the system. Existing approaches
to robot formation control generally fall into three categories: behavior based [5], virtual structure
[6] and leader following [7].
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
22
In the leader-follower approach, a robot of the formation, designed as the leader, moves along a
predefined trajectory while the other robots, the followers, are required to maintain a desired
posture (distance and orientation) with respect to the leader [7]. The primary advantage with the
approach is that it reduces the formation control problem to a tracking problem where stability of
the tracking error can be obtained through standard control theoretic techniques.
One way of realizing formation control in this approach is to shift the domain of the computation
space into the vision domain of the follower so that the formation control problem is derived in
the local image plane of the follower only undergoing planar motion. The image processing
algorithms are then made in real time. using suitable hardware. There has been many attempts to
develop fast and simple image analysis methods in real time using a set of visual markers or a set
of feature points in real time. The relative pose is then obtained using a geometric pose estimation
technique. The fastest object detection in real time is provided by the Viola Jones algorithm using
a cascades of classifiers based on Haar-like features . The features is robust to motion blur and
changing lighting conditions because those features represent basic elements of the picture
including edges, lines and dots. The cascades are trained using AdaBoost learning and calculated
with a very fast using integral image Result shows that the system can detect pedestrian at 17
frames per second on 1.7 GHz processor with pedestrian detection rate up to 90% of accuracy [9].
As an alternative, in this paper, we implement for the real time imaging and processing, a color
and texture based algorithms developed for vision detection and localization that are also simple
but effective and are similar to approaches used for many computer vision tasks such as face
tracking and detection.The object detection system is generalized for the formation control
problem using vision, and is capable of handling partially occluded vehicles, improves the
sensing conditions by accounting for lighting and environmental conditions in addition to real
time implementation. In this the traditional categories of algorithms for tracking, namely methods
based on target localization and representation are judiciously combined with methods based on
filtering and data association.For on-line tracking the Continuously Adapted Mean Shift
(CAMShift) [10], [11] algorithm is used which allows on-time tracking. Kalman filters are used
to keep track of the object location and predict the location of the object in subsequent frames to
help the CAMShift algorithm locate the target in the next. A second filter tracks the leader as it
passes under occlusions by using the velocity and position of the object as it becomes occluded to
maintain a search region for the CAMShift function as the target reappears from the occlusion. A
third filter tracks the area returned by the CAMShift algorithm and monitors changes in area to
detect occlusions early.
2. PREVIOUS WORK
The purpose of this paper is vision based tracking control in a leader follower formation. It
includes occlusion handling, lighting and environmental considerations. Hence we review these
state of art detection domains Considerable work has been done in visual tracking to overcome
the difficulties arising from noise, occlusion, clutter and changes.
In general tracking algorithms fall into two categories: a) methods based on filtering and data
association, b) methods based on target representation and localization.
Tracking algorithms relying on target representation and localization are based on mostly
measurements. They employ a model of the object appearance and try to detect this model in
consecutive frames of the image sequence. Color or texture features of the object, are used to
create a histogram. The object's position is estimated by minimizing a cost function between the
model's histogram and candidate histograms in the next image. An example of the method in this
category is the mean shift algorithm where the object inside an ellipse and the histogram is
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
23
constructed from pixel values inside that ellipse [12]. The extensions of the main algorithm are
proposed in [13]. The mean shift is combined with particle filters in [14]. Scale invariant features
are used in [15] where various distance measures are associated with the mean shift algorithm.
These methods have the drawback that the type of object's movement should be correctly
modeled.
The algorithms based on filtering assume that the moving object has an internal state which may
be measured and, by combining the measurements with the model of state evolution, the object's
position is estimated. An example of this category is the Kalman filter [16] which successfully
tracks objects even in the case of occlusion [17], the particle filters [13,14] the Condensation [18]
and ICondensation [19] algorithms which are more general than Kalman filters These algorithms.
have the ability to predict an object's location under occlusion as well and use factored sampling.
3. PRESENTED APPROACH FOR TRACKING USING THE VISION SYSTEM
The presented work combines in real time the advantages of pose representation through color
and texture, localization through the CAMShift algorithm and uses the filtering and data
association problem through Kalman filtering for occlusion detection The visual tracking problem
is divided into target detection and pose estimation problem The overall flow diagram of the
proposed approach for target detection is described in Fig.1. The color histogram matching is
used to get a robust target identification which is fed to CAMShift algorithm for robust tracking.
In a continuous incoming video sequence, the change of the location of the object leads to
dynamic changes of the probability distribution of the target. CAMShift changes the probability
distribution dynamically, and adjusts the size and location of the searching window based on the
change of probability distribution. The back projected image is given as input for CAMShift
processing for tracking the target. After each frame is read, the target histogram is updated to
account for any illumination or color shift changes of the target. This is done by computing a
candidate histogram of the target in the same manner the original target histogram is computed.
Then the candidate and target histograms are compared using the histogram comparison
technique that uses a Bhattacharyya Coefficient [20]. The Bhattacharyya Coefficient returns a
value of 0 or 1, with 0 being a perfect match and 1 being a total mismatch of the histograms.
Once a region of interest (ROI) is measured, the algorithm creates a vector that shifts the focus of
the tracker to the new centre of density.
3.1. Object representation using color
Color is used in identifying and isolating objects. The RGB values of every pixel in the frame are
read and converted into the HSL color space. The HSL color space is chosen because the
chromatic information is independent from the lighting conditions. Hue specifies the base color,
saturation determines the intensity of the color and luminance is dependent on the lighting
condition. Since every color has its own range of H values, the program compares the H values of
every pixel with a predefined range of H values of the landing zone. If it falls within 10%, the
pixel is marked out as being part of the landing zone. With the range of H correctly chosen, the
landing spot is identified more accurately. After going through all the pixels for one frame, the
centroid of all the marked pixels is calculated. From this result, the relative position of the landing
zone with respect to the center of the camera’s view is known. The conversion from RGB color
space to HSV color space is performed using equation (1).Here red (r), green (g), blue (b) ∈[0,1]
are the coordinates of the RGB color space and max and minicorrespond to the greatest and least
of r, g and b respectively .The Hue angle h∈[0,360] for HSV color space is given by [21].
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
24
(1)
The value of h is normalized to lie between 0 and 180଴
to fit into an 8 bit gray scale image (0-
255), and h= 0 is used when max=min, though the hue has no geometric meaning for gray. The s
and v values for HSV color space are defined as follows:
(2)
The v or value channel represents the gray scale portion of the image. A threshold for the Hue
value of the image is set based on the mounted marker color. Using the threshold value,
segmentation between the desired color information and other colors is performed. The resulting
image is a binary image with white indicating the desired color region ad black assumed to be the
noisy region. The contour of desired region is obtained as described in the section below.
3.2 Contour detection
Freeman chain code [22] method is used for finding contours, which is based on 8 connectivity of
3x3 windows of Freeman chain code. Two factors determine the success of the algorithm: The
first factor is the direction of traverses either clockwise or anticlockwise. The other is the start
location of the 3X3 window traverse. Chain code scheme is a representation that consists of series
of numbers which represent the direction of the next pixel that can be used to represent shape of
the objects. Chain code is a list of codes ranging from 0 to 7 in clockwise direction representing
the direction of the next pixel connected in 3X3 windows as shown in Fig. 2. The coordinate of
the next pixel is calculated based on the addition and subtraction of columns and rows by 1,
depending on the value of the chain code.
3.3 Background projection
A back projection image is obtained using the Hue, Saturation and local binary pattern channels
along with the target histogram. The back projection image is a mono channel image whose pixel
value probability range between 0to255 that corresponds to the probability of the pixel values in
the ROI.
The histogram process categorizes the value of each pixel in the selected region and assigns each
into one of N bins, corresponding to N bins of the histogram dimension. In this case a three-
dimensional histogram is used with dimension of 32 (hue) X 16 (saturation) x 36 (LBP) bins =
18,432 bins. In a similar way, a histogram is created for the remainder of the background to
identify the predominant values not in the target. Weights are assigned for each of the target bins
such that the target values unique to the target will have a higher relative value versus hues that
are in common with the background. The resulting histogram lookup table maps the input image
plane value to the bin count, and then normalizes from 0 to 255 to ensure the resulting grey scale
image has valid pixel intensity values. In the paper, the initial target histogram lookup table is
created at target selection and is saved as a reference histogram for later processing. The latest

















=+
−
−
=+
−
−
=+
−
−
=
=
bif
gr
gif
rb
rif
bg
if
h
max,240
minmax
*60
,max,120
minmax
*60
,max,360
minmax
*60
min,max,0
οο
οο
οο





−=
−
=
=
otherwise
if
s
,
max
min
1
max
minmax
,0max,0
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
25
target histogram is then used for each frame to map the hue, saturation image planes into a
resulting back projection image used by the CAMShift process. The resulting image from this
process has mostly white pixels (255) at the predominant target locations and lower values for the
remaining colors.
Fig. 1- Flowchart of the Vision Algorithm
Figure 1. Flowchart of the Vision Algorithm
Figure 2. Freeman Chain code representation
3.4 Tracking using CAMShift
The inputs to the CAMShift algorithm are window coordinates to restrict the search, and a back
projection image where the pixel values represent the relative probability that the pixel contains
the hues, saturation. CAMShift algorithm operates on the probability distribution image that is
derived from the histogram of the object to be tracked generate above.
The principle steps of the CAMShift algorithm are as follows:
1) Choose the initial location of the mean shift search window.
2) Calculate the 2D color histogram within the search window.
3) Perform back-projection of the histogram to a region of interest (ROI) centered at the search
window but slightly larger the mean shift window size.
5 6 7
4 Pixel
Position
X,Y
0
3 2 1
Current pixel position x, y
Code Next row Next column
0 X y+1
1 x-1 y+1
2 x-1 Y
3 x-1 y-1
4 X y-1
5 x+1 y-1
6 x+1 Y
7 x+1 y+1
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
26
4) Iterate Mean Shift algorithm to find the centroid of the probability image and store the zeroth
moment and centroid location. The mean location within the search window of the discrete
probability
(3)
image is found using moments. Given that I(x, y) is the intensity of the discrete probability image
at (x, y) within the search window, the zeroth moment is computed as:(3)
The first Moment for x and y is,
(4)
Then the mean search window location can be found as:
(5)
For the next video frame, center the search window at the mean location stored in Step 4 and set
the window’s size to a function of the zeroth
moment. Go to Step 2. The scale of the target is
determined by finding an equivalent rectangle that has the same moments as those measured from
the probability distribution image. Define the second moments as:
(6)
The following intermediate variables are used
(7)
Then the dimension of the search window can be computed as:
Figure 3. Detected pattern on Follower Image
∑ ∑=
x y
yxIM ),(0,0
∑ ∑=
∑ ∑=
x y
yxxIM
x y
yxyIM
),(0,1
),(1,0








=
00
01,
00
10),(
M
M
M
M
cycx
∑ ∑=
∑ ∑=
∑ ∑=
x y
yxxyIM
x y
yxIyM
x y
yxIxM
),(1,1
),(2
2,0
),(2
0,2













−=
−=
−=
y
M
M
c
xy
M
M
b
cx
M
M
a
0,0
2,0
0,0
1,1
2
0,0
0,2
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
27
(8)
3.5 Pose Estimation
The image measurements from the follower are the relative distance and the relative bearing in
the leader follower framework. The follower vehicle is equipped with a forward facing camera
mounted on it. Camera pose calculation from one frame to the other requires information about
objects that are viewable in the frames. Information for this is obtained from the feature
extraction method proposed above. The camera captures features from the pattern mounted on the
leader robot, as shown in Fig.3.The four markers, are converted to high intensity values in the
HSV channels and localized from frame to frame using the CAMShift-Back ground projection
filtering process.
From these feature positions, the relative posture of the follower with respect to leader is
determined. The equations are
(9)
Where E is the length of the sides of the square. Using inverse perspective projection the
measurement equations is formulated as shown in Fig. 4.
(10)
Using variables of equations (6) and (7), and assuming the robot centre and camera centre
coincide, the relative position and angular displacement with respect to the target vehicle (ϕ ,D)
can be calculated using the following equations.
Figure 4. Target position with respect to camera centre







 −
−=
+
=
+
=
E
L
x
R
x
T
LzRz
TzLxRx
Tx 1cos,
2
,
2
ϕ
















+=+=
−
=
−
=
1,1
,
Rh
E
fRz
Lh
E
fLz
Bx
f
fRz
RxAx
f
fLz
Lx
2
2)(2)(
2
2)(2)(
cabca
w
cabca
h
−+++
=
−+−+
=
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
28





























+=
+=
−
−−=
−=
)22(
1tan
1tan
TZTxD
LXRX
LZRZ
TZ
Tx
φϕθ
φ
ϕ
(11)
3.6 Vehicle Control
In our case, we have two robots; the tasks of each robot are different and based on their role.
Leader has as primary task to track a pre-defined trajectory using a robust controller based on
sliding mode control [23]. Follower uses a visual feedback and tries to identify and track the
trajectory of the leader. Control laws for the follower vehicle are based on incorporating range
and bearing to the leader vehicle that is received from the camera. Based on the range between
the leader and the follower vehicle, and line of sight guidance definitions as shown in [24] with
reference to Fig. 4 the follower vehicle adjusts its speed (U) in order to achieve the command
range
As shown in Fig.5 , (X(i-1), Y(i-1) ) , (X(i), Y(i) ) and (X(i+1), Y(i+1) ) are the ((i-1)th
, ith
and
(i+1)th
waypoints (denoted as ‘o’) respectively assigned by the mission plan. The LOS guidance
employs the line of sight between the vehicle and the targets The LOS angle to the next waypoint
is
defined as
)(
)(
arctan
txiX
tyiY
los −
−
=ψ (12)
where inertial positions are ( ))(),(),(( itytx dψ .
The angle of current line of tracking is
)1(
1arctan)(
−−
−−
=
iXiX
iYiY
idψ (13)
The cross track error is given by:
)()()(~
)(ker tit dirorcrosstrac ψψψ −= (14)
The distance from current to the next waypoint is:
22
)(
~
)(
~
)( iii tYtXtS += (15)
Where the cross track error is given by:
)()(
))(sin()()(
itd
tdtSt
dlosp
pi
ψψ
ε
−=
=
(16)
Range error is defined as:
(17)
To determine the speed of the follower vehicle, the following formulas are defined [24]:
)sgn( zz
σησ −=
•
(18)
comStSS −= )(
~
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
29
where zz
~=σ Also,
)0(~ −==
•
Szz
&&σ (19)
Figure 5. Track Geometry used for Line of Sight Guidance
Where is defined by
(20)
Where,
x଴ is the leader's x-position
y଴ is the leader's y-position
x is the follower’s x-position
y is the follower’s y-position
(21)
(22)
Combining Equations (21) and (22) and solving for U results in the following formula:
(23)
The follower vehicle adjusts its heading ( fψ ) in order to achieve the command bearing ( β com)
depending upon the bearing between the leader and the follower vehicle. Bearing error is defined
as:
( )comt βββ −= )(
~
(24)
In order to determine the heading of the follower vehicle the following formulas are defined:
(25)
(26)
•
S
)0(0)0(0 •
−
•−
+
•
−
•−
=
•
















yy
z
yy
xx
z
xx
S
)0sin(00 ψUy =&
)sin(ψUy =&
)0(sin)0(cos
)0(0)0(0)~sgn(
yyxx
yyyxxxzz
U
−+−
−+−+−
=
ψψ
&&
( )
( )rλβηδ
βλβηβσ
+−=
+−=
~
sgn
~~
sgn &&
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
30
4. RESULTS
To test the effectiveness of the vision algorithms, experiments have been consisting of two
ground vehicles in a convoy formation using autonomous control for the master and vision
control for the slave robot which uses a low cost vision sensor(camera) as the only sensor to
obtain the location information. The robot vehicle frame and chassis is a model manufactured by
Robokit. It has a differential 4-wheeled robot base with two wheels at the front and the two
wheels are at the back. The vehicle uses four DC motors (geared) for driving the four wheels
independently. The DC motors are controlled by a microcontroller. For the vision algorithm the
coding is performed in python language with OpenCV library modules ported to python block.
Practical experiments were conducted in order to test the performance and robustness of the
proposed vision based target following which are described in the forth coming sections.
Figure 6. Occlusion detection and recovery of target vehicle in real time
4.1 Partially and fully occluded target detection
Figure 6 illustrates the case where the target is partially (sequence 2) and fully occluded
(sequence 3) while moving through an occlusion (wooden stool) .The recovery from occlusion is
illustrated in Fig. 6 (sequence 4). This process was repeated for multiple environments, in real-
time, and the algorithm works robustly for all of the used tests, showing the overall effectiveness
of this algorithm. The recovery was found to be accurate in all cases experimentally studied.
Target Detection in presence of same/similar color and texture Fig. 8 demonstrates the efficacy of
the detection process using the traditional CAMShift using color based features.
.
Figure 7: Target tracking with similar colored object using traditional CAMShift
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
31
As can be seen from the video sequence of Fig. 8, because the hues are similar and are of similar
texture to the tracked markers of the leader vehicle, the tracker leaves the banner (marker on the
leader vehicle) and remains locked to the background object. Fig.9 shows the improvement to the
detection process obtained by adding the saturation and texture functions of the leader vehicle in
the presence of similar color/textured objects in the background. The hue + saturation + texture
method works very well in tracking the leader through the background that is of a nearly identical
color as the leader red marker. There is enough difference in texture that the back projection
algorithm is able to create a probability density function that correctly isolates the leader from the
background
Figure 8. Target tracking with similar colored background using proposed CAMShift+ LBP + Kalman
4.2 Illumination Variation
Fig. 10 highlights the improvements made by the addition of saturation and texture channels to
the back projection process when compared to the CAMShift algorithm using hue only (Figure 9)
The improvement on how the leader is isolated from the similar illuminated background is clearly
evident.
Figure 9. Target tracking response with indoor light variation using CAMShift
Figure 10. Target tracking response with indoor light variation using CAMShift+LBP+Kalman
5. CONCLUSIONS
The algorithms developed for vision detection and localization provide a simple but effective
processing approach in real time for the leader-follower formation control approach considered in
the paper. The generalized framework proposed using the Vision Algorithm presented based on
Hue+ Saturation + texture when combined with Kalman filters accounts for full and partial
occlusion of the leader, the lighting and environmental conditions and overcomes practical
limitations that exist in the information gathering process in the leader follower framework. The
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
32
CAMShift tracker based on hue only becomes confused when the background and the object have
similar hues, as it is unable to differentiate the two. Selection of appropriate Saturation and
Luminance thresholds and implementation in a weighted histogram reduces the impact of
common hues and enhances the performance detections for the CAMShift tracker in the leader
follower framework. The addition of Local Binary Patterns greatly increases track performance.
The addition of the dynamic updating of the target histogram to accounts for lighting improves
the track for long sequences or sequences with varying lighting conditions. The controller using
line of sight guidance for the follower faithfully approaches the leader by maintaining the speed
and distance as demanded, indicating the validity of the approach and method used.
REFERENCES
[1] J. Jennings, G. Whelan and Evans .W (1997). “Cooperative search and rescue with a team of mobile
robots,”ICAR ’97 Proceedings, 8th International Conference on Advanced Robotics 193–200.
[2] Dietl .M,. Gutmann. J.-S and Nebel. B (2001), Cooperative Sensing in Dynamic Environments,
IEEE/RSJ Int. Conf. on Intelligent Robots and Systems.
[3] Renaud. P, Cervera .E, and Martiner. P (2004) “Towards a reliable vision- based mobile robot
formation control”, Intelligent Robots and Systems. Proceedings. IEEE/RSJ International Conference
on 4:176–3181.
[4] H. Yamaguchi, (1997)“Adaptive formation control for distributed autonomous mobile robot groups,”
in Proc. IEEE Int. Conf. Robotics and Automation,Albuquerque, NM, Apr., 2300–2305.
[5] T. Balch and R. C. Arkin,(1998) “Behavior-based formation control for multirobot teams,” IEEE
Trans. Robot. Automat., 14, 926–939.
[6] R. W. Beard, J. Lawton, and F. Y. Hadaegh, (2001) “A feedback architecture for formation control,”
IEEE Trans. Control Syst. Technol.,. 9, 777–790.
[7] P. K. C. Wang, “Navigation strategies for multiple autonomous mobile robots moving in formation,”
(1991) J. Robot. Syst.,. 8, 177–195,
[8] Viola, P. & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features, 1, I–
511–I–518.
[9] Fiala. M (2004). “Vision Guided Control of Multiple Robots,” Proceedings of the First Canadian
Conference on Computer and Robot Vision, Washington DC, USA, 241-246.
[10] Fukunaga F., Hostetler L. D (1975). “The Estimation of the Gradient of a Density Function, with
Applications in Pattern Recognition”, IEEE Trans. on Information Theory, 21, (1), 32-40.
[11] Bradski G(1998).“Computer Vision Face Tracking for Use in a Perceptual User Interface”, Intel
Technology Journal, 2(Q2), 1-15.
[12] Comaniciu. D , Ramesh. V, Meer. P (2003). Kernel-based object tracking, IEEE Transactions on
Pattern Analysis and Machine Intelligence, 25 (5) 564–577.
[13] Zhoo. H, Yuan. Y, Shi. C (2009). “Object tracking using SIFT features and mean Shift”, Computer
Vision and Image Understanding 113 (3) 345– 352.
[14] Yang. C, Duraiswami. R, Davis. L (2005). “Efficient mean-shift tracking via a new similarity
measure” , Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern
Recognition (CVPR05), 1, 176–183.
[15] Fan. Z, Yang. M, Wu. Y (2007).” Multiple, Collaborative kernel tracking”, IEEE Transactions on
Pattern Analysis and Machine Intelligence 29 (7) 1268–1273.
[16] Simon. D, Kalman, H (2006). “Optimal State Estimation: and Non Linear Approaches”, Wiley–
Interscience.
[17] Yang. M, Wu. Y, Hua. G (2009). “Context-aware visual tracking”, IEEE Transactions on Pattern
Analysis and Machine Intelligence 31 (7) 1195–1209.
[18] Isard. M, Blake. A (1998). “Condensation—conditional density propagation for visual tracking”,
International Journal of Computer Vision 29, 5– 28.
[19] M. Isard, A. Blake, (1998) Icondensation—unifying low-level and high-level tracking in a stochastic
framework, Proceedings of the 5th European Conference on Computer Vision (ECCV), 893–908
[20] A. Bhattacharayya, On a measure of divergence between two statistical populations defined by their
probability distributions. Bulletin of the Calcutta Mathematical Society 35, 99-110.
[21] Achim Zeileis, Kurt Hornik, Paul Murrell (2009) Escaping RGBland: Selecting colors for statistical
graphics Computational Statistics and data Analysis 53, 3259-3270.
The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014
33
[22] A. Akimov, A. Kolesnikov, and P. Fränti (2007), “Lossless compression of map contours by context
tree modeling of chain codes,” Pattern Recognition, 40 944–952.
[23] F.A. Papoulias (1995). “Non-Linear Dynamics and Bifurcations in Autonomous Vehicle Guidance
and Control”, Underwater Robotic Vehicles: Design and Control, (Ed. J. Yuh), Published by TST
Press, Albuquerque, NM. ISBN #0-6927451-6-2, pp. 41-72.
[24] D.B. Marco AND A.J. Healey (2001). ”Command, Control and Navigation experimental results with
Aries AUV”, IEEE Journal of Oceanic Engineering” 26, NO. 4.

More Related Content

PDF
MULTIPLE OBJECTS TRACKING IN SURVEILLANCE VIDEO USING COLOR AND HU MOMENTS
PDF
Uncalibrated View Synthesis Using Planar Segmentation of Images
PDF
Ijcatr02011007
PDF
International Journal of Engineering Research and Development (IJERD)
PDF
B04410814
PDF
K-Means Clustering in Moving Objects Extraction with Selective Background
PDF
Vehicle detection and tracking techniques a concise review
PDF
Object extraction using edge, motion and saliency information from videos
MULTIPLE OBJECTS TRACKING IN SURVEILLANCE VIDEO USING COLOR AND HU MOMENTS
Uncalibrated View Synthesis Using Planar Segmentation of Images
Ijcatr02011007
International Journal of Engineering Research and Development (IJERD)
B04410814
K-Means Clustering in Moving Objects Extraction with Selective Background
Vehicle detection and tracking techniques a concise review
Object extraction using edge, motion and saliency information from videos

What's hot (18)

PDF
CLASSIFICATION AND COMPARISON OF LICENSE PLATES LOCALIZATION ALGORITHMS
PDF
CLASSIFICATION AND COMPARISON OF LICENSE PLATES LOCALIZATION ALGORITHMS
PDF
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
PDF
ramya_Motion_Detection
PDF
Multiple Ant Colony Optimizations for Stereo Matching
PDF
A METHOD FOR TRACKING ROAD OBJECTS
PDF
IRJET-Vision Based Occupant Detection in Unattended Vehicle
PDF
3 d mrf based video tracking in the compressed domain
PDF
3 d mrf based video tracking in the compressed domain
PDF
3-d interpretation from stereo images for autonomous driving
PDF
D018112429
PDF
Camera-Based Road Lane Detection by Deep Learning II
PDF
EFFECTIVE REDIRECTING OF THE MOBILE ROBOT IN A MESSED ENVIRONMENT BASED ON TH...
PDF
EFFECTIVE REDIRECTING OF THE MOBILE ROBOT IN A MESSED ENVIRONMENT BASED ON TH...
PDF
Effective Object Detection and Background Subtraction by using M.O.I
PPT
Video Surveillance Systems For Traffic Monitoring
PDF
Medial Axis Transformation based Skeletonzation of Image Patterns using Image...
PDF
IRJET- Automatic Traffic Sign Detection and Recognition using CNN
CLASSIFICATION AND COMPARISON OF LICENSE PLATES LOCALIZATION ALGORITHMS
CLASSIFICATION AND COMPARISON OF LICENSE PLATES LOCALIZATION ALGORITHMS
IJCER (www.ijceronline.com) International Journal of computational Engineerin...
ramya_Motion_Detection
Multiple Ant Colony Optimizations for Stereo Matching
A METHOD FOR TRACKING ROAD OBJECTS
IRJET-Vision Based Occupant Detection in Unattended Vehicle
3 d mrf based video tracking in the compressed domain
3 d mrf based video tracking in the compressed domain
3-d interpretation from stereo images for autonomous driving
D018112429
Camera-Based Road Lane Detection by Deep Learning II
EFFECTIVE REDIRECTING OF THE MOBILE ROBOT IN A MESSED ENVIRONMENT BASED ON TH...
EFFECTIVE REDIRECTING OF THE MOBILE ROBOT IN A MESSED ENVIRONMENT BASED ON TH...
Effective Object Detection and Background Subtraction by using M.O.I
Video Surveillance Systems For Traffic Monitoring
Medial Axis Transformation based Skeletonzation of Image Patterns using Image...
IRJET- Automatic Traffic Sign Detection and Recognition using CNN
Ad

Similar to Leader follower formation control of ground vehicles using camshift based guidance (20)

PDF
F124144
PDF
COMPARATIVE STUDY ON VEHICLE DETECTION TECHNIQUES IN AERIAL SURVEILLANCE
PDF
Vehicle Tracking Using Kalman Filter and Features
PDF
Robot Pose Estimation: A Vertical Stereo Pair Versus a Horizontal One
PDF
A robust ga knn based hypothesis
PDF
Real Time Object Identification for Intelligent Video Surveillance Applications
PDF
MODEL BASED TECHNIQUE FOR VEHICLE TRACKING IN TRAFFIC VIDEO USING SPATIAL LOC...
PDF
Identification and classification of moving vehicles on road
PDF
Leader Follower Formation Control of Ground Vehicles Using Dynamic Pixel Coun...
PDF
VISUAL TRACKING USING PARTICLE SWARM OPTIMIZATION
PDF
VISUAL TRACKING USING PARTICLE SWARM OPTIMIZATION
PDF
Visual tracking using particle swarm optimization
PDF
proceedings of PSG NCIICT
PDF
Speed Determination of Moving Vehicles using Lucas- Kanade Algorithm
PDF
In tech vision-based_obstacle_detection_module_for_a_wheeled_mobile_robot
PDF
A New Algorithm for Tracking Objects in Videos of Cluttered Scenes
PDF
A ROS IMPLEMENTATION OF THE MONO-SLAM ALGORITHM
PPTX
traffic jam detection using image processing
PDF
A survey on moving object tracking in video
PDF
Ijecet 06 10_003
F124144
COMPARATIVE STUDY ON VEHICLE DETECTION TECHNIQUES IN AERIAL SURVEILLANCE
Vehicle Tracking Using Kalman Filter and Features
Robot Pose Estimation: A Vertical Stereo Pair Versus a Horizontal One
A robust ga knn based hypothesis
Real Time Object Identification for Intelligent Video Surveillance Applications
MODEL BASED TECHNIQUE FOR VEHICLE TRACKING IN TRAFFIC VIDEO USING SPATIAL LOC...
Identification and classification of moving vehicles on road
Leader Follower Formation Control of Ground Vehicles Using Dynamic Pixel Coun...
VISUAL TRACKING USING PARTICLE SWARM OPTIMIZATION
VISUAL TRACKING USING PARTICLE SWARM OPTIMIZATION
Visual tracking using particle swarm optimization
proceedings of PSG NCIICT
Speed Determination of Moving Vehicles using Lucas- Kanade Algorithm
In tech vision-based_obstacle_detection_module_for_a_wheeled_mobile_robot
A New Algorithm for Tracking Objects in Videos of Cluttered Scenes
A ROS IMPLEMENTATION OF THE MONO-SLAM ALGORITHM
traffic jam detection using image processing
A survey on moving object tracking in video
Ijecet 06 10_003
Ad

Recently uploaded (20)

PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PPTX
TLE Review Electricity (Electricity).pptx
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PPTX
Spectroscopy.pptx food analysis technology
PDF
Network Security Unit 5.pdf for BCA BBA.
PDF
Encapsulation theory and applications.pdf
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
Heart disease approach using modified random forest and particle swarm optimi...
PDF
Machine learning based COVID-19 study performance prediction
PDF
A comparative analysis of optical character recognition models for extracting...
PPTX
A Presentation on Artificial Intelligence
PPTX
SOPHOS-XG Firewall Administrator PPT.pptx
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
PPTX
Programs and apps: productivity, graphics, security and other tools
PPTX
Machine Learning_overview_presentation.pptx
PDF
MIND Revenue Release Quarter 2 2025 Press Release
PDF
NewMind AI Weekly Chronicles - August'25-Week II
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
TLE Review Electricity (Electricity).pptx
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Spectroscopy.pptx food analysis technology
Network Security Unit 5.pdf for BCA BBA.
Encapsulation theory and applications.pdf
Reach Out and Touch Someone: Haptics and Empathic Computing
Heart disease approach using modified random forest and particle swarm optimi...
Machine learning based COVID-19 study performance prediction
A comparative analysis of optical character recognition models for extracting...
A Presentation on Artificial Intelligence
SOPHOS-XG Firewall Administrator PPT.pptx
Digital-Transformation-Roadmap-for-Companies.pptx
Group 1 Presentation -Planning and Decision Making .pptx
Programs and apps: productivity, graphics, security and other tools
Machine Learning_overview_presentation.pptx
MIND Revenue Release Quarter 2 2025 Press Release
NewMind AI Weekly Chronicles - August'25-Week II

Leader follower formation control of ground vehicles using camshift based guidance

  • 1. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 DOI : 10.5121/ijma.2014.6602 21 LEADER FOLLOWER FORMATION CONTROL OF GROUND VEHICLES USING CAMSHIFT BASED GUIDANCE S.M.Vaitheeswaran, Bharath.M.K, and Gokul.M Aerospace Electronics and Systems Division, CSIR-National Aerospace laboratories, HAL Airport Road, Kodihalli, Bengaluru, 560017 India ABSTRACT Autonomous ground vehicles have been designed for the purpose of that relies on ranging and bearing information received from forward looking camera on the Formation control . A visual guidance control algorithm is designed where real time image processing is used to provide feedback signals. The vision subsystem and control subsystem work in parallel to accomplish formation control. A proportional navigation and line of sight guidance laws are used to estimate the range and bearing information from the leader vehicle using the vision subsystem. The algorithms for vision detection and localization used here are similar to approaches for many computer vision tasks such as face tracking and detection that are based color-and texture based features, and non-parametric Continuously Adaptive Mean-shift algorithms to keep track of the leader. This is being proposed for the first time in the leader follower framework. The algorithms are simple but effective for real time and provide an alternate approach to traditional based approaches like the Viola Jones algorithm. Further to stabilize the follower to the leader trajectory, the sliding mode controller is used to dynamically track the leader. The performance of the results is demonstrated in simulation and in practical experiments. KEYWORDS Continuously Adapted Mean Shift, Local Binary Pattern (LBP), Kalman filtering, leader-follower, formation control. 1. INTRODUCTION The problem of controlling a formation of ground vehicles of autonomous ground vehicles continues to draw the attention of researches, thanks to its applications in complex tasks like search and rescue [1], mapping of unknown environments [2], and mobile robotics [3].By formation control we simply mean the problem of controlling the relative position and orientation of the vehicles in a group while allowing the group to move as a whole [4]. The use of formation control enables one to achieve complex tasks with a low robot complexity since each particular task can be specialized in one task of manipulation or robot navigation. Perception, path generation and control characteristics, generally decide the formation control process for their robustness. Also the system architectures provide the infrastructure upon which formation control is implemented and determines the capabilities and limitations of the system. Existing approaches to robot formation control generally fall into three categories: behavior based [5], virtual structure [6] and leader following [7].
  • 2. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 22 In the leader-follower approach, a robot of the formation, designed as the leader, moves along a predefined trajectory while the other robots, the followers, are required to maintain a desired posture (distance and orientation) with respect to the leader [7]. The primary advantage with the approach is that it reduces the formation control problem to a tracking problem where stability of the tracking error can be obtained through standard control theoretic techniques. One way of realizing formation control in this approach is to shift the domain of the computation space into the vision domain of the follower so that the formation control problem is derived in the local image plane of the follower only undergoing planar motion. The image processing algorithms are then made in real time. using suitable hardware. There has been many attempts to develop fast and simple image analysis methods in real time using a set of visual markers or a set of feature points in real time. The relative pose is then obtained using a geometric pose estimation technique. The fastest object detection in real time is provided by the Viola Jones algorithm using a cascades of classifiers based on Haar-like features . The features is robust to motion blur and changing lighting conditions because those features represent basic elements of the picture including edges, lines and dots. The cascades are trained using AdaBoost learning and calculated with a very fast using integral image Result shows that the system can detect pedestrian at 17 frames per second on 1.7 GHz processor with pedestrian detection rate up to 90% of accuracy [9]. As an alternative, in this paper, we implement for the real time imaging and processing, a color and texture based algorithms developed for vision detection and localization that are also simple but effective and are similar to approaches used for many computer vision tasks such as face tracking and detection.The object detection system is generalized for the formation control problem using vision, and is capable of handling partially occluded vehicles, improves the sensing conditions by accounting for lighting and environmental conditions in addition to real time implementation. In this the traditional categories of algorithms for tracking, namely methods based on target localization and representation are judiciously combined with methods based on filtering and data association.For on-line tracking the Continuously Adapted Mean Shift (CAMShift) [10], [11] algorithm is used which allows on-time tracking. Kalman filters are used to keep track of the object location and predict the location of the object in subsequent frames to help the CAMShift algorithm locate the target in the next. A second filter tracks the leader as it passes under occlusions by using the velocity and position of the object as it becomes occluded to maintain a search region for the CAMShift function as the target reappears from the occlusion. A third filter tracks the area returned by the CAMShift algorithm and monitors changes in area to detect occlusions early. 2. PREVIOUS WORK The purpose of this paper is vision based tracking control in a leader follower formation. It includes occlusion handling, lighting and environmental considerations. Hence we review these state of art detection domains Considerable work has been done in visual tracking to overcome the difficulties arising from noise, occlusion, clutter and changes. In general tracking algorithms fall into two categories: a) methods based on filtering and data association, b) methods based on target representation and localization. Tracking algorithms relying on target representation and localization are based on mostly measurements. They employ a model of the object appearance and try to detect this model in consecutive frames of the image sequence. Color or texture features of the object, are used to create a histogram. The object's position is estimated by minimizing a cost function between the model's histogram and candidate histograms in the next image. An example of the method in this category is the mean shift algorithm where the object inside an ellipse and the histogram is
  • 3. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 23 constructed from pixel values inside that ellipse [12]. The extensions of the main algorithm are proposed in [13]. The mean shift is combined with particle filters in [14]. Scale invariant features are used in [15] where various distance measures are associated with the mean shift algorithm. These methods have the drawback that the type of object's movement should be correctly modeled. The algorithms based on filtering assume that the moving object has an internal state which may be measured and, by combining the measurements with the model of state evolution, the object's position is estimated. An example of this category is the Kalman filter [16] which successfully tracks objects even in the case of occlusion [17], the particle filters [13,14] the Condensation [18] and ICondensation [19] algorithms which are more general than Kalman filters These algorithms. have the ability to predict an object's location under occlusion as well and use factored sampling. 3. PRESENTED APPROACH FOR TRACKING USING THE VISION SYSTEM The presented work combines in real time the advantages of pose representation through color and texture, localization through the CAMShift algorithm and uses the filtering and data association problem through Kalman filtering for occlusion detection The visual tracking problem is divided into target detection and pose estimation problem The overall flow diagram of the proposed approach for target detection is described in Fig.1. The color histogram matching is used to get a robust target identification which is fed to CAMShift algorithm for robust tracking. In a continuous incoming video sequence, the change of the location of the object leads to dynamic changes of the probability distribution of the target. CAMShift changes the probability distribution dynamically, and adjusts the size and location of the searching window based on the change of probability distribution. The back projected image is given as input for CAMShift processing for tracking the target. After each frame is read, the target histogram is updated to account for any illumination or color shift changes of the target. This is done by computing a candidate histogram of the target in the same manner the original target histogram is computed. Then the candidate and target histograms are compared using the histogram comparison technique that uses a Bhattacharyya Coefficient [20]. The Bhattacharyya Coefficient returns a value of 0 or 1, with 0 being a perfect match and 1 being a total mismatch of the histograms. Once a region of interest (ROI) is measured, the algorithm creates a vector that shifts the focus of the tracker to the new centre of density. 3.1. Object representation using color Color is used in identifying and isolating objects. The RGB values of every pixel in the frame are read and converted into the HSL color space. The HSL color space is chosen because the chromatic information is independent from the lighting conditions. Hue specifies the base color, saturation determines the intensity of the color and luminance is dependent on the lighting condition. Since every color has its own range of H values, the program compares the H values of every pixel with a predefined range of H values of the landing zone. If it falls within 10%, the pixel is marked out as being part of the landing zone. With the range of H correctly chosen, the landing spot is identified more accurately. After going through all the pixels for one frame, the centroid of all the marked pixels is calculated. From this result, the relative position of the landing zone with respect to the center of the camera’s view is known. The conversion from RGB color space to HSV color space is performed using equation (1).Here red (r), green (g), blue (b) ∈[0,1] are the coordinates of the RGB color space and max and minicorrespond to the greatest and least of r, g and b respectively .The Hue angle h∈[0,360] for HSV color space is given by [21].
  • 4. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 24 (1) The value of h is normalized to lie between 0 and 180଴ to fit into an 8 bit gray scale image (0- 255), and h= 0 is used when max=min, though the hue has no geometric meaning for gray. The s and v values for HSV color space are defined as follows: (2) The v or value channel represents the gray scale portion of the image. A threshold for the Hue value of the image is set based on the mounted marker color. Using the threshold value, segmentation between the desired color information and other colors is performed. The resulting image is a binary image with white indicating the desired color region ad black assumed to be the noisy region. The contour of desired region is obtained as described in the section below. 3.2 Contour detection Freeman chain code [22] method is used for finding contours, which is based on 8 connectivity of 3x3 windows of Freeman chain code. Two factors determine the success of the algorithm: The first factor is the direction of traverses either clockwise or anticlockwise. The other is the start location of the 3X3 window traverse. Chain code scheme is a representation that consists of series of numbers which represent the direction of the next pixel that can be used to represent shape of the objects. Chain code is a list of codes ranging from 0 to 7 in clockwise direction representing the direction of the next pixel connected in 3X3 windows as shown in Fig. 2. The coordinate of the next pixel is calculated based on the addition and subtraction of columns and rows by 1, depending on the value of the chain code. 3.3 Background projection A back projection image is obtained using the Hue, Saturation and local binary pattern channels along with the target histogram. The back projection image is a mono channel image whose pixel value probability range between 0to255 that corresponds to the probability of the pixel values in the ROI. The histogram process categorizes the value of each pixel in the selected region and assigns each into one of N bins, corresponding to N bins of the histogram dimension. In this case a three- dimensional histogram is used with dimension of 32 (hue) X 16 (saturation) x 36 (LBP) bins = 18,432 bins. In a similar way, a histogram is created for the remainder of the background to identify the predominant values not in the target. Weights are assigned for each of the target bins such that the target values unique to the target will have a higher relative value versus hues that are in common with the background. The resulting histogram lookup table maps the input image plane value to the bin count, and then normalizes from 0 to 255 to ensure the resulting grey scale image has valid pixel intensity values. In the paper, the initial target histogram lookup table is created at target selection and is saved as a reference histogram for later processing. The latest                  =+ − − =+ − − =+ − − = = bif gr gif rb rif bg if h max,240 minmax *60 ,max,120 minmax *60 ,max,360 minmax *60 min,max,0 οο οο οο      −= − = = otherwise if s , max min 1 max minmax ,0max,0
  • 5. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 25 target histogram is then used for each frame to map the hue, saturation image planes into a resulting back projection image used by the CAMShift process. The resulting image from this process has mostly white pixels (255) at the predominant target locations and lower values for the remaining colors. Fig. 1- Flowchart of the Vision Algorithm Figure 1. Flowchart of the Vision Algorithm Figure 2. Freeman Chain code representation 3.4 Tracking using CAMShift The inputs to the CAMShift algorithm are window coordinates to restrict the search, and a back projection image where the pixel values represent the relative probability that the pixel contains the hues, saturation. CAMShift algorithm operates on the probability distribution image that is derived from the histogram of the object to be tracked generate above. The principle steps of the CAMShift algorithm are as follows: 1) Choose the initial location of the mean shift search window. 2) Calculate the 2D color histogram within the search window. 3) Perform back-projection of the histogram to a region of interest (ROI) centered at the search window but slightly larger the mean shift window size. 5 6 7 4 Pixel Position X,Y 0 3 2 1 Current pixel position x, y Code Next row Next column 0 X y+1 1 x-1 y+1 2 x-1 Y 3 x-1 y-1 4 X y-1 5 x+1 y-1 6 x+1 Y 7 x+1 y+1
  • 6. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 26 4) Iterate Mean Shift algorithm to find the centroid of the probability image and store the zeroth moment and centroid location. The mean location within the search window of the discrete probability (3) image is found using moments. Given that I(x, y) is the intensity of the discrete probability image at (x, y) within the search window, the zeroth moment is computed as:(3) The first Moment for x and y is, (4) Then the mean search window location can be found as: (5) For the next video frame, center the search window at the mean location stored in Step 4 and set the window’s size to a function of the zeroth moment. Go to Step 2. The scale of the target is determined by finding an equivalent rectangle that has the same moments as those measured from the probability distribution image. Define the second moments as: (6) The following intermediate variables are used (7) Then the dimension of the search window can be computed as: Figure 3. Detected pattern on Follower Image ∑ ∑= x y yxIM ),(0,0 ∑ ∑= ∑ ∑= x y yxxIM x y yxyIM ),(0,1 ),(1,0         = 00 01, 00 10),( M M M M cycx ∑ ∑= ∑ ∑= ∑ ∑= x y yxxyIM x y yxIyM x y yxIxM ),(1,1 ),(2 2,0 ),(2 0,2              −= −= −= y M M c xy M M b cx M M a 0,0 2,0 0,0 1,1 2 0,0 0,2
  • 7. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 27 (8) 3.5 Pose Estimation The image measurements from the follower are the relative distance and the relative bearing in the leader follower framework. The follower vehicle is equipped with a forward facing camera mounted on it. Camera pose calculation from one frame to the other requires information about objects that are viewable in the frames. Information for this is obtained from the feature extraction method proposed above. The camera captures features from the pattern mounted on the leader robot, as shown in Fig.3.The four markers, are converted to high intensity values in the HSV channels and localized from frame to frame using the CAMShift-Back ground projection filtering process. From these feature positions, the relative posture of the follower with respect to leader is determined. The equations are (9) Where E is the length of the sides of the square. Using inverse perspective projection the measurement equations is formulated as shown in Fig. 4. (10) Using variables of equations (6) and (7), and assuming the robot centre and camera centre coincide, the relative position and angular displacement with respect to the target vehicle (ϕ ,D) can be calculated using the following equations. Figure 4. Target position with respect to camera centre         − −= + = + = E L x R x T LzRz TzLxRx Tx 1cos, 2 , 2 ϕ                 +=+= − = − = 1,1 , Rh E fRz Lh E fLz Bx f fRz RxAx f fLz Lx 2 2)(2)( 2 2)(2)( cabca w cabca h −+++ = −+−+ =
  • 8. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 28                              += += − −−= −= )22( 1tan 1tan TZTxD LXRX LZRZ TZ Tx φϕθ φ ϕ (11) 3.6 Vehicle Control In our case, we have two robots; the tasks of each robot are different and based on their role. Leader has as primary task to track a pre-defined trajectory using a robust controller based on sliding mode control [23]. Follower uses a visual feedback and tries to identify and track the trajectory of the leader. Control laws for the follower vehicle are based on incorporating range and bearing to the leader vehicle that is received from the camera. Based on the range between the leader and the follower vehicle, and line of sight guidance definitions as shown in [24] with reference to Fig. 4 the follower vehicle adjusts its speed (U) in order to achieve the command range As shown in Fig.5 , (X(i-1), Y(i-1) ) , (X(i), Y(i) ) and (X(i+1), Y(i+1) ) are the ((i-1)th , ith and (i+1)th waypoints (denoted as ‘o’) respectively assigned by the mission plan. The LOS guidance employs the line of sight between the vehicle and the targets The LOS angle to the next waypoint is defined as )( )( arctan txiX tyiY los − − =ψ (12) where inertial positions are ( ))(),(),(( itytx dψ . The angle of current line of tracking is )1( 1arctan)( −− −− = iXiX iYiY idψ (13) The cross track error is given by: )()()(~ )(ker tit dirorcrosstrac ψψψ −= (14) The distance from current to the next waypoint is: 22 )( ~ )( ~ )( iii tYtXtS += (15) Where the cross track error is given by: )()( ))(sin()()( itd tdtSt dlosp pi ψψ ε −= = (16) Range error is defined as: (17) To determine the speed of the follower vehicle, the following formulas are defined [24]: )sgn( zz σησ −= • (18) comStSS −= )( ~
  • 9. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 29 where zz ~=σ Also, )0(~ −== • Szz &&σ (19) Figure 5. Track Geometry used for Line of Sight Guidance Where is defined by (20) Where, x଴ is the leader's x-position y଴ is the leader's y-position x is the follower’s x-position y is the follower’s y-position (21) (22) Combining Equations (21) and (22) and solving for U results in the following formula: (23) The follower vehicle adjusts its heading ( fψ ) in order to achieve the command bearing ( β com) depending upon the bearing between the leader and the follower vehicle. Bearing error is defined as: ( )comt βββ −= )( ~ (24) In order to determine the heading of the follower vehicle the following formulas are defined: (25) (26) • S )0(0)0(0 • − •− + • − •− = •                 yy z yy xx z xx S )0sin(00 ψUy =& )sin(ψUy =& )0(sin)0(cos )0(0)0(0)~sgn( yyxx yyyxxxzz U −+− −+−+− = ψψ && ( ) ( )rλβηδ βλβηβσ +−= +−= ~ sgn ~~ sgn &&
  • 10. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 30 4. RESULTS To test the effectiveness of the vision algorithms, experiments have been consisting of two ground vehicles in a convoy formation using autonomous control for the master and vision control for the slave robot which uses a low cost vision sensor(camera) as the only sensor to obtain the location information. The robot vehicle frame and chassis is a model manufactured by Robokit. It has a differential 4-wheeled robot base with two wheels at the front and the two wheels are at the back. The vehicle uses four DC motors (geared) for driving the four wheels independently. The DC motors are controlled by a microcontroller. For the vision algorithm the coding is performed in python language with OpenCV library modules ported to python block. Practical experiments were conducted in order to test the performance and robustness of the proposed vision based target following which are described in the forth coming sections. Figure 6. Occlusion detection and recovery of target vehicle in real time 4.1 Partially and fully occluded target detection Figure 6 illustrates the case where the target is partially (sequence 2) and fully occluded (sequence 3) while moving through an occlusion (wooden stool) .The recovery from occlusion is illustrated in Fig. 6 (sequence 4). This process was repeated for multiple environments, in real- time, and the algorithm works robustly for all of the used tests, showing the overall effectiveness of this algorithm. The recovery was found to be accurate in all cases experimentally studied. Target Detection in presence of same/similar color and texture Fig. 8 demonstrates the efficacy of the detection process using the traditional CAMShift using color based features. . Figure 7: Target tracking with similar colored object using traditional CAMShift
  • 11. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 31 As can be seen from the video sequence of Fig. 8, because the hues are similar and are of similar texture to the tracked markers of the leader vehicle, the tracker leaves the banner (marker on the leader vehicle) and remains locked to the background object. Fig.9 shows the improvement to the detection process obtained by adding the saturation and texture functions of the leader vehicle in the presence of similar color/textured objects in the background. The hue + saturation + texture method works very well in tracking the leader through the background that is of a nearly identical color as the leader red marker. There is enough difference in texture that the back projection algorithm is able to create a probability density function that correctly isolates the leader from the background Figure 8. Target tracking with similar colored background using proposed CAMShift+ LBP + Kalman 4.2 Illumination Variation Fig. 10 highlights the improvements made by the addition of saturation and texture channels to the back projection process when compared to the CAMShift algorithm using hue only (Figure 9) The improvement on how the leader is isolated from the similar illuminated background is clearly evident. Figure 9. Target tracking response with indoor light variation using CAMShift Figure 10. Target tracking response with indoor light variation using CAMShift+LBP+Kalman 5. CONCLUSIONS The algorithms developed for vision detection and localization provide a simple but effective processing approach in real time for the leader-follower formation control approach considered in the paper. The generalized framework proposed using the Vision Algorithm presented based on Hue+ Saturation + texture when combined with Kalman filters accounts for full and partial occlusion of the leader, the lighting and environmental conditions and overcomes practical limitations that exist in the information gathering process in the leader follower framework. The
  • 12. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 32 CAMShift tracker based on hue only becomes confused when the background and the object have similar hues, as it is unable to differentiate the two. Selection of appropriate Saturation and Luminance thresholds and implementation in a weighted histogram reduces the impact of common hues and enhances the performance detections for the CAMShift tracker in the leader follower framework. The addition of Local Binary Patterns greatly increases track performance. The addition of the dynamic updating of the target histogram to accounts for lighting improves the track for long sequences or sequences with varying lighting conditions. The controller using line of sight guidance for the follower faithfully approaches the leader by maintaining the speed and distance as demanded, indicating the validity of the approach and method used. REFERENCES [1] J. Jennings, G. Whelan and Evans .W (1997). “Cooperative search and rescue with a team of mobile robots,”ICAR ’97 Proceedings, 8th International Conference on Advanced Robotics 193–200. [2] Dietl .M,. Gutmann. J.-S and Nebel. B (2001), Cooperative Sensing in Dynamic Environments, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems. [3] Renaud. P, Cervera .E, and Martiner. P (2004) “Towards a reliable vision- based mobile robot formation control”, Intelligent Robots and Systems. Proceedings. IEEE/RSJ International Conference on 4:176–3181. [4] H. Yamaguchi, (1997)“Adaptive formation control for distributed autonomous mobile robot groups,” in Proc. IEEE Int. Conf. Robotics and Automation,Albuquerque, NM, Apr., 2300–2305. [5] T. Balch and R. C. Arkin,(1998) “Behavior-based formation control for multirobot teams,” IEEE Trans. Robot. Automat., 14, 926–939. [6] R. W. Beard, J. Lawton, and F. Y. Hadaegh, (2001) “A feedback architecture for formation control,” IEEE Trans. Control Syst. Technol.,. 9, 777–790. [7] P. K. C. Wang, “Navigation strategies for multiple autonomous mobile robots moving in formation,” (1991) J. Robot. Syst.,. 8, 177–195, [8] Viola, P. & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features, 1, I– 511–I–518. [9] Fiala. M (2004). “Vision Guided Control of Multiple Robots,” Proceedings of the First Canadian Conference on Computer and Robot Vision, Washington DC, USA, 241-246. [10] Fukunaga F., Hostetler L. D (1975). “The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition”, IEEE Trans. on Information Theory, 21, (1), 32-40. [11] Bradski G(1998).“Computer Vision Face Tracking for Use in a Perceptual User Interface”, Intel Technology Journal, 2(Q2), 1-15. [12] Comaniciu. D , Ramesh. V, Meer. P (2003). Kernel-based object tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, 25 (5) 564–577. [13] Zhoo. H, Yuan. Y, Shi. C (2009). “Object tracking using SIFT features and mean Shift”, Computer Vision and Image Understanding 113 (3) 345– 352. [14] Yang. C, Duraiswami. R, Davis. L (2005). “Efficient mean-shift tracking via a new similarity measure” , Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05), 1, 176–183. [15] Fan. Z, Yang. M, Wu. Y (2007).” Multiple, Collaborative kernel tracking”, IEEE Transactions on Pattern Analysis and Machine Intelligence 29 (7) 1268–1273. [16] Simon. D, Kalman, H (2006). “Optimal State Estimation: and Non Linear Approaches”, Wiley– Interscience. [17] Yang. M, Wu. Y, Hua. G (2009). “Context-aware visual tracking”, IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (7) 1195–1209. [18] Isard. M, Blake. A (1998). “Condensation—conditional density propagation for visual tracking”, International Journal of Computer Vision 29, 5– 28. [19] M. Isard, A. Blake, (1998) Icondensation—unifying low-level and high-level tracking in a stochastic framework, Proceedings of the 5th European Conference on Computer Vision (ECCV), 893–908 [20] A. Bhattacharayya, On a measure of divergence between two statistical populations defined by their probability distributions. Bulletin of the Calcutta Mathematical Society 35, 99-110. [21] Achim Zeileis, Kurt Hornik, Paul Murrell (2009) Escaping RGBland: Selecting colors for statistical graphics Computational Statistics and data Analysis 53, 3259-3270.
  • 13. The International Journal of Multimedia & Its Applications (IJMA) Vol.6, No.6, December 2014 33 [22] A. Akimov, A. Kolesnikov, and P. Fränti (2007), “Lossless compression of map contours by context tree modeling of chain codes,” Pattern Recognition, 40 944–952. [23] F.A. Papoulias (1995). “Non-Linear Dynamics and Bifurcations in Autonomous Vehicle Guidance and Control”, Underwater Robotic Vehicles: Design and Control, (Ed. J. Yuh), Published by TST Press, Albuquerque, NM. ISBN #0-6927451-6-2, pp. 41-72. [24] D.B. Marco AND A.J. Healey (2001). ”Command, Control and Navigation experimental results with Aries AUV”, IEEE Journal of Oceanic Engineering” 26, NO. 4.