Object Tracking using Artificial Neural Network

Modeling object tracking using artificial neural
networks
Jeff Ames Anwar Jameel Yuhua Ni∗
December 9, 2015
Abstract
Tracking objects with the eyes is an essential skill for most animals,
yet how this feat is achieved is still relatively poorly understood, even
disregarding complications due to binocular vision and object tracking in
three dimensions. In this paper, we present a model using an artificial
neural network (ANN) that seeks to mimic simple object tracking in two
dimensions. We show that ANNs are capable of performing this task, and
we propose future directions along which to continue this research.
1 Introduction
Object tracking is the problem of determining how to move one’s body, head,
and eyes, in order to keep a target in view. It is crucial for any number of
common tasks, such as hunting, foraging, and obstacle avoidance, and should
be heavily selected for in evolution.
One approach to object tracking is using an Artificial Neural Network (ANN).
The ANN architecture is designed to mimic the information processing struc-
ture of the human brain. Where the brain contains neurons with dendrites,
axons, and synapses, the ANN contains computational units receiving inputs
and outputs. Unlike conventional computational tools, ANNs do not use stan-
dard decision rules and algorithms instructed by the user to produce an output
from an input. Instead, this system can improve its own rules and decisions the
more it is used and trained. Since this system makes use of parallel comput-
ing and dynamically learns to improve its decision making, ANNs have become
very useful for a variety of purposes such as stock market predictions, air-line
security control, and investment management. [1]
Compared to classical approaches, ANNs are much faster in their ability to
adapt towards models of greater precision using fewer experimental data sets.
One effective way of using ANNs for object tracking is through the global pixel-
based approach. In this approach, the ANN takes in as input, the entirety of
the desired object’s pixel representation instead of looking for patterns in local
correspondences or using other characteristics from the appearance of the image.
This allows for recognition of more complex elements in an image in exchange
for longer run-times. There have been successful cases of object tracking using
∗{jca105, aj528, yn85}@rutgers.edu
1

this approach and it has shown promise in its ability to recognize both static
and dynamic objects.[2] Furthermore, this approach has also been shown to be
effective in the field of computer vision and shows potential in aiding to develop
more autonomous robots. [3]
2 Background
2.1 Artificial neural networks
The most commonly used model of ANN is the multilayered perceptron model
(MLP), as shown in Figure 1. In this model, the structure is set up so that
each neuron output is connected to every neuron in the subsequent layer such
that it forms a cascade with no connections between neurons in the same layer.
Overall, the popular MLP model has been introduced into various industries for
uses in motors, sensors, chemical process systems, and more. This is also the
model that we are using in our tests for object tracking. [4][5]
Figure 1: Typical MLP ANN[4]
To determine a neuron’s output, first the weighted inputs and bias are
summed:
a = Σn
i=1wixi − b
This value is then passed through a sigmoid function (see Figure 2), which
notably has two nonlinear areas around the central linear range of operation, to
obtain the neuron’s output:
y =
1
1 + e−a
2

0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
−10 −5 0 5 10
Figure 2: Sigmoid function
2.2 Eye movement
Figure 3: Extra-ocular muscles (courtesy of cnx.org/contents/14fb4ad7-39a1-
4eee-ab6e-3ef2482e3e22@6.27)
The extraocular muscles are the six muscles that control movement of the eye
and one muscle that controls eyelid elevation (levator palpebrae). The actions
of the six muscles responsible for eye movement depend on the position of the
eye at the time of muscle contraction.
3 Model
We model the visual ﬁeld as a discrete 20 × 20 array and consider the upper
left corner to be position (1, 1). The eyes by default are set to focus on squares
(7, 11) and (14, 11). We then present a stimulus at the center of one of these
grid squares.
3

(a) Visual field with eye focal points at
default positions
(b) Visual field with eyes starting to
track stimulus
To track the stimulus, we employ a 3-layer neural network – one layer for
inputs, one hidden layer, and one output layer. Each layer is fully wired to the
next, and each connection has a weight associated with it. Each neuron also has
a bias input of -1 to allow the network to shift the sigmoid function left or right.
Figure 5 shows an abbreviated diagram of the ANN used for object tracking.
In the actual network, we had 6 inputs (each position has a separate x and y
component), we used a hidden layer consisting of 20 neurons, and there were 4
outputs (left/right and up/down directions for each eye).
Figure 5: Simplified version of our artificial neural network
The inputs to the network are the coordinates of the stimulus and the current
focal points of the left and right eyes. The network outputs two values per eye,
indicating the x and y direction the eye should move to focus on the target.
The network is trained using back-propagation on a small training set (4000
samples out of a parameter space of size 6.4 × 107
) until an error less than
0.0005 is achieved (where error for each eye is mean squared error between the
direction chosen by the network and the actual direction from current position
to the stimulus). It is also verified against a test set (another 4000 samples)
to ensure the network is not over-trained. We use the pyfann library (python
bindings for the libfann library) to create, train, and test our network.
4

Note that the stimulus position is discrete, but the eye positions are treated
as ﬂoating point values during simulation. However, the training and test sets
contain only discrete values for both the the stimulus and eye positions.
4 Results
0
5
10
15
20
0 5 10 15 20
y
x
Figure 6: Eye location (green and blue dots) for a given stimulus position (red)
Interactive use of the network shows that it is able to realistically track a stim-
ulus, given the current eye position. Figure 6 shows the eye positions (green for
left, blue for right) as the stimulus position is varied over all points in the 20
× 20 visual grid. In all central cases, the points almost coincide. For points on
the far left or far right, only one eye is capable of focusing on it, and the other
eye goes as far as it can, forming the vertical bands at x = 4.5 and x = 14.5.
5

−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
0 200 400 600 800 1000 1200 1400
Left eye X
Left eye Y
Right eye X
Right eye Y
Figure 7: Eye velocities when moving towards a stimulus
Figure 7 shows how the eyes move to follow a stimulus. The stimulus was
first positioned in the upper right, then to the four corners in counter-clockwise
order. The flat sections of the graph (e.g., at t = 200) represent times when the
eyes have come to focus on the stimulus. The stimulus was then moved, causing
the subsequent spike in velocity.
We also note that the ANN was successfully able to generalize from discrete
to real-valued eye positions. In the training and test data sets, all stimulus and
eye positions were restricted to the 20 × 20 discrete grid in the visual field, but
in simulation we treat the eye positions as floating point numbers. The fact that
the eyes continue to smoothly track the stimulus implies that the neural network
is able to generalize and interpolate correct outputs for these novel positions.
5 Conclusion
We have shown that an artificial neural network is capable of reproducing
human-like object tracking behavior, given a relatively sparse set of data to learn
from. In the future, we’d like to extend our model to a fully three-dimensional
model, and we expect that ANNs will continue to perform well in a 3D domain.
We’d also like to test a similar model using a biologically-realistic spiking neu-
ron model. This would be more difficult to set up and train, but it would have
many benefits in terms of modeling behavioral pathologies, such as lazy eye,
that the ANN model fails to capture.
6

6 Acknowledgments
We thank our advisor, Prof. Konstantinos Michmizos, for many useful sugges-
tions throughout the duration of this project.
References
[1] E. Y. Li, “Artificial neural networks and their business applications,” Infor-
mation & Management, vol. 27, no. 5, pp. 303–313, 1994.
[2] M. B. M. Bouzenada and Z. Telli, “Neural Network for Object Tracking,”
Information Technology Journal, vol. 6, no. 4, pp. 526–533, 2007.
[3] J. Leitner, S. Harding, M. Frank, A. Förster, and J. Schmidhuber, “Artificial
Neural Networks For Spatial Perception,” International Joint Conference on
Neural Networks, 2013.
[4] M. R. G. Meireles, P. E. M. Almeida, and M. G. Simões, “A comprehen-
sive review for industrial applicability of artificial neural networks,” IEEE
Transactions on Industrial Electronics, vol. 50, no. 3, pp. 585–601, 2003.
[5] C. M. Bishop, Neural networks for pattern recognition. Oxford university
press, 1995.
7

Object Tracking using Artificial Neural Network

More Related Content

What's hot (17)

Viewers also liked (20)

Similar to Object Tracking using Artificial Neural Network (20)

Recently uploaded (20)

Object Tracking using Artificial Neural Network