Mak ms

SWARM ALGORITHMS:
SIMULATION AND GENERATION
by
MICHAEL ANTHONY KOVACINA
Submitted in partial fulﬁllment of the requirements
For the degree of Master of Science
Thesis Advisers:
Dr. Michael S. Branicky
Dr. Daniel W. Palmer
Dr. Ravi Vaidyanathan
Department of Electrical Engineering and Computer Science
CASE WESTERN RESERVE UNIVERSITY
January, 2006

Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Summary and Outline . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Swarm Intelligence 6
2.1 Swarm Intelligence Overview . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Deﬁnition of Swarm Intelligence . . . . . . . . . . . . . . . . . . . . . 8
2.3 Swarm Intelligence as a Problem Solving Technique . . . . . . . . . . 10
2.4 Applications of Swarm Intelligence . . . . . . . . . . . . . . . . . . . 11
3 Swarm Simulation Software 16
3.1 Existing Swarm Simulation Software . . . . . . . . . . . . . . . . . . 16
3.2 Design Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.3 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4 Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.5 Avatar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.6 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.7 Probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4 SWEEP Applications 27
4.1 Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2 Task Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.3 Chemical Cloud Detection with UAV Swarms . . . . . . . . . . . . . 33
4.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.3.2 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.3.3 Search Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 36
4.3.4 Cloud Detection and Mapping . . . . . . . . . . . . . . . . . . 38
4.3.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5 Evolutionary Generation of Swarm Algorithms 47
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.2 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.3 System Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.4 Solution Representation . . . . . . . . . . . . . . . . . . . . . . . . . 51
i

5.5 Evolutionary Operators . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.6 Fitness Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
6 Results of Evolving Swarm Behaviors 56
6.1 Swarm Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.1.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.1.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.1.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.2 Object Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.2.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.2.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
7 Conclusion 95
7.1 SWEEP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7.2 Autogeneration of Swarm Algorithms through Evolution . . . . . . . 96
7.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
A Example SWEEP Simulation Speciﬁcation 100
B Evolved Object Destruction Solution 107
C Evolved Object Collection Solution 116
D Evolved Object Manipulation Solution 120
ii

List of Figures
1.1 A screenshot of SWEEP executing a swarm algorithm . . . . . . . . . 3
3.1 High-level overview of the SWEEP architecture. . . . . . . . . . . . . 19
3.2 A pseudo specification file for a SWEEP simulation. . . . . . . . . . . 20
3.3 A simple example of a finite state machine. . . . . . . . . . . . . . . . 22
3.4 State-based XML format of the finite state machine in Figure 3.3. . . 22
3.5 Transition-based XML format of the finite state machine in Figure 3.3. 22
4.1 An example of the dispersion algorithm in action . . . . . . . . . . . 28
4.2 A 125 agent swarm executing the dispersion algorithm. . . . . . . . . 29
4.3 Results from an example CAST auction. . . . . . . . . . . . . . . . . 32
4.4 The different stages of a chemical cloud detection mission: deployment,
detection, and mapping. . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.5 Postulated UAV flight model . . . . . . . . . . . . . . . . . . . . . . . 36
4.6 Examples of symmetrical search algorithms for UAVs. . . . . . . . . . 38
4.7 Examples of inside-outside and dispersion cloud mapping algorithms
for UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.8 Cloud mapping results for a swarm of 10 UAVs. . . . . . . . . . . . . 42
4.9 The normalized amount of time taken by a UAV swarm to locate a
chemical cloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.10 The percentage of trials where the UAV swarm successfully detects the
chemical cloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.11 The average hit percentage of a UAV swarm of size n for any sized cloud. 46
4.12 The average amount of time taken by a UAV swarm of size n to find
any sized cloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.1 Conceptual overview of ECS (Evolutionary Computing for Swarms). 48
5.2 A small section of a state machine solution in the native XML format. 52
5.3 Graphical depictions of mutation operator functionality . . . . . . . . 54
6.1 Dispersion criteria for a single agent. . . . . . . . . . . . . . . . . . . 58
6.2 Best and average fitness scores resulting from evolving a dispersion
algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.3 A simple example of radix sorting. . . . . . . . . . . . . . . . . . . . . 70
6.4 Progress of the best solution over 50 generations for object destruction. 79
6.5 Mean fitness over time for object destruction. . . . . . . . . . . . . . 79
iii

6.6 Progress of the best solution over time for object collection. . . . . . 84
6.7 Mean mean fitness for object collection. . . . . . . . . . . . . . . . . . 84
6.8 Progress of the best solution for object manipulation. . . . . . . . . . 91
6.9 Mean fitness over time for object manipulation. . . . . . . . . . . . . 91
6.10 Progress of the best solution for another object manipulation evolution. 92
6.11 Mean fitness over time for the second object manipulation evolution. 92
6.12 The fitness of every solution indexed by generation with respect to
metric m5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.13 The fitness of every solution indexed by generation with respect to
metric m7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.14 Number of solutions satisfying criteria m1, m2, m3, and m4 for object
manipulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
iv

List of Tables
2.1 Examples of swarm intelligence that can be found throughout nature. 7
2.2 Benefits of using a swarm intelligence as a problem solving technique. 12
4.1 Summary of dispersion simulation parameters . . . . . . . . . . . . . 30
4.2 Summary of CAST Auction simulation parameters . . . . . . . . . . 32
5.1 An example set of parameters for evolving dispersion. . . . . . . . . . 50
5.2 Mutations defined for a SWEEP state machine solution encoding . . 53
6.1 Parameters for evolving dispersion . . . . . . . . . . . . . . . . . . . . 60
6.2 Evolved dispersion state machine . . . . . . . . . . . . . . . . . . . . 62
6.3 Object manipulation behavior primitives available for state machine
construction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.4 Object manipulation sensor primitives available for constructing state
transitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.5 Basic fitness metrics for an object manipulation scenario. . . . . . . . 68
6.6 Composite metrics used for the object manipulation scenario. . . . . . 70
6.7 Parameters for evolving object collection, destruction, and manipulation 72
6.8 Evolved object destruction state machine . . . . . . . . . . . . . . . . 77
6.9 Simplified version of the evolved object destruction state machine . . 78
6.10 Evolved object collection algorithm . . . . . . . . . . . . . . . . . . . 83
6.11 Simplified version of the evolved object collection algorithm . . . . . 83
6.12 Evolved object manipulation algorithm . . . . . . . . . . . . . . . . . 89
6.13 Simplified version of the evolved object manipulation algorithm . . . 90
v

Acknowledgments
There are many people that I would like to thank. First I would like to thank my
family for enabling and encouraging me in all of my life’s pursuits. To my grandma,
for putting up with my “experiments” in her kitchen when I was younger. To my
grandpa, for the musical training that gave me the arts to complement my sciences.
To my brother. . . yes, I’m thanking you. To my dad, for teaching me about working
hard and having patience. To my mom, for teaching me to do things the right way.
And to my wife, your confidence and patience (especially the patience) mean the
world to me.
Next, I would like to thank Dr. Dan Palmer for giving me the opportunity to
do swarm research while I was an undergraduate at John Carroll University. Also,
I want to thank Dr. Palmer for all of his advice and guidance, both academically
and personally. In the same vein, I want to thank Dr. Palmer and Chad Hantak for
having the inspiration and vision to develop the first incarnation of SWEEP.
I would also like to thank Orbital Research for providing me with the opportunity
to get my Masters. Throughout this process, there have been many supportive people
at Orbital, too many to thank right now. But suffice it to say, thank you all for
bettering me as a writer and a researcher, giving me a sounding board for my ideas,
and providing me with coffee and encouragement.
I would like to express my thanks to my thesis advisers, Dr. Michael Branicky, Dr.
Dan Palmer, and Dr. Ravi Vaidyanathan, for all their time and effort, especially in
the last few weeks of this process. Their guidance and input truly helped me navigate
this endeavor. And to the rest of my committee, Dr. Randy Beer and Dr. Roger
Quinn, thank you for taking time out of your schedules to read my thesis and be on
my committee.
vi

Swarm Algorithms:
Simulation and Generation
Abstract
by
MICHAEL ANTHONY KOVACINA
This work implements and demonstrates a set of tools to automatically generate
swarm algorithms. SWEEP, SWarm Experimentation and Evaluation Platform, a
swarm algorithm development and simulation platform is implemented. To demon-
strate the utility of SWEEP, several swarm-based algorithms are constructed for
free-moving and dynamics-constrained agents, culminating in the development of ba-
sic searching and mapping algorithms for unmanned air vehicles tracking chemical
clouds. ECS, Evolutionary Computing for Swarms, is implemented to evolve swarm
algorithms that exhibit emergent behavior. ECS represents solutions as ﬁnite state
machines, utilizes SWEEP to simulate a swarm executing each state machine solu-
tion, and employs radix-based ranking to address the multi-objective nature of the
problem. As a demonstration of the ability of ECS, swarm algorithms for agent
dispersion, object collection, and object destruction are evolved and discussed.

Chapter 1
Introduction
This thesis implements and demonstrates a set of tools to automatically generate
swarm algorithms. SWEEP (SWarm Experimentation and Evaluation Platform), a
swarm algorithm development and simulation platform is implemented. To demon-
strate the utility of SWEEP, several swarm-based algorithms are constructed for
free-moving and dynamics-constrained agents, culminating in the development of ba-
sic searching and mapping algorithms for UAVs (unmanned air vehicles) tracking
chemical clouds. ECS (Evolutionary Computing for Swarms), is implemented to
evolve swarm algorithms that exhibit emergent behavior. ECS represents solutions
as ﬁnite state machines, utilizes SWEEP to simulate a swarm executing each state
machine solution, and employs radix-based ranking to address the multi-objective na-
ture of the problem. As a demonstration of the ability of ECS, swarm algorithms for
agent dispersion, object collection, and object destruction are evolved and discussed.
1.1 Motivation
Flocking birds, foraging ants, aggregating slime molds: these are just a few of the
striking examples of self-organizing behaviors that are observed in nature. Through
a solely decentralized and reactive system, where “the whole is greater than the sum
1

of the parts,” swarms exhibit behaviors that coordinate on a global scale. Deriving
inspiration from nature, swarm theory leverages the self-organization that emerges
from the interactions of multiple agents to evoke a group-level behavior that is beyond
the capability of any single member of the swarm. Swarm algorithms are most useful
for problems that are amenable to an agent-based decomposition, have a dynamic
nature, and do not require time-limited or optimal solutions.
Perhaps the most advantageous aspect of swarm intelligence is the large body
of work from which to draw inspiration [46]. Originally, swarm theory looked to
societal insects such as ants for inspiration. For example, the method in which an
ant colony will determine the closest of multiple food sources is also applicable to
addressing graph-based problems such as the traveling salesman problem and the
minimum spanning tree problem [7]. But as more attention is focused on swarm
intelligence, other sources of inspiration are identiﬁed. Recent research is focused
on extracting the implicit swarm algorithms that arise from the interactions of more
intelligent organisms, i.e., humans [39].
As the size of the swarm and the complexity of the tasks increase, the opportunities
for emergence increase dramatically. In order to fully exploit the power of swarm
algorithms, human programmers will need tools that extend beyond those currently
available for traditional programming. These tools will enable the programmer to
work and think on a higher-level than the single agent, specifying goals and constraints
for the entire swarm as opposed to loops and conditionals. This is analogous to the
development of high-level languages like C and FORTRAN that allowed programmers
to focus more on concepts rather implementation. Additionally, tools will be required
for the simulation of the developed algorithm for the purposes of evaluation and
testing. Finally, tools to identify and classify emergence in a swarm will be required
to optimize and verify the performance of a swarm algorithm.
2

1.2 Contributions
This thesis demonstrates the use of evolutionary computing for the successful autogen-
eration of swarm algorithms. In order to construct swarm algorithms, a development
environment is required to facilitate experimentation and evaluation, hence the con-
struction of a SWarm Experimentation and Evaluation Platform (SWEEP). SWEEP
is designed to be open and extensible. The core SWEEP architecture is constructed
with minimal assumptions regarding agent modeling, controller design, or environ-
mental parameters. Figure 1.1 is a screenshot of SWEEP executing a demonstration
swarm algorithm. The utility of SWEEP is demonstrated through the simulation of
several swarm-based scenarios including agent dispersion, task assignment, chemical
cloud detection, and object manipulation.
Figure 1.1: A screenshot of SWEEP executing a swarm algorithm
There is a small body of work related to the evolution of swarm algorithms, mostly
related to the development of behaviors for swarm robotics systems. Zaera et al. [54]
used an evolutionary method to develop behaviors for aggregation, dispersal, and
schooling in fish. Though successful in developing aggregation and dispersal algo-
rithms, disappointing results were realized with regards to the schooling behavior,
mainly due to the difficulty involved in crafting a fitness evaluation function that ad-
equately captured the salient features of a schooling behavior. Gaudiano, Bonabeau,
3

and Shargel [17] used a traditional genetic algorithm to tune probabilities for hand-
made probabilistic state machines used to coordinate the behaviors of a swarm of
unmanned air vehicles. Perhaps the best known example of using evolution to gen-
erate swarm behaviors is the work performed by Koza. In Koza’s work [31], he used
stateless LISP S-expressions to evolve an ant-like food foraging algorithm for a small
swarm of simulated ants.
This thesis extends the body of work related to the generation of swarm behaviors
through evolutionary programming. In contrast to Koza’s work which used LISP S-
expressions to generate state-less algorithms, this thesis represents candidate solutions
as finite state machines to provide for the evolution of state-based algorithms. The
evolutionary computation system developed uses the SWEEP platform to simulate
candidate solutions and evaluate their fitness with respect to the multiple goals of
the target scenario. Additionally, a unique data fusion method is employed along
with radix-based ranking to overcome the challenges presented by the multi-objective
fitness functions required to evolve correct swarm algorithms.
1.3 Summary and Outline
This thesis focuses on the identification and implementation of tools required for the
autogeneration of swarm algorithms. The work done in this thesis is organized into
seven chapters.
Chapter 1 lays out the focus of this thesis, the motivation for researching the auto-
generation of swarm algorithms, and the contributions made by this work.
Chapter 2 introduces the concept of swarm intelligence. In addition, the term
swarm intelligence is formally defined for this work.
Chapter 3 presents SWEEP and its design.
4

Chapter 4 describes several example swarm algorithms developed with SWEEP in-
cluding dispersion, task assignment, and chemical cloud detection using UAVs.
Chapter 5 presents ECS, the evolutionary computing system used to evolve the
swarm algorithms in Chapter 6. Here we also discuss using SWEEP as a fitness
function.
Chapter 6 explores the evolution of swarm algorithms for agent dispersion and ob-
ject manipulation, a combination of object collection and object destruction.
The fitness measures used for each scenario are discussed. In addition, evolved
solutions for object manipulation are presented.
Chapter 7 reviews the conclusions and contributions of this work, and suggests
avenues for future research efforts.
5

Chapter 2
Swarm Intelligence
2.1 Swarm Intelligence Overview
Birds flock, fish school, slime molds aggregate. Why is it that evolution has produced
swarming in so many different contexts? The formation of a swarm simultaneously
provides the individual and the group a number of benefits arising from the synergy
of interaction. As an agent in the swarm, the potential benefits available to the
individual is maximized through the efforts of the others in the swarm. From the
perspective of the swarm as an organism, the survivability of the swarm is increased
through the higher-level coordination that emerges from the low-level interactions of
the individuals.
There are several examples in nature that exhibit the individual and group benefits
of a swarm. First, swarming provides the ability to forage more effectively. Perhaps
the best known example is the foraging behavior of ants. Each ant searches for food,
and upon finding a food source returns to the colony, emitting a pheromone trail
along the way. Other ants detect the pheromone trail and follow the trail to the
food source. In turn, they also emit the pheromone on the return trip to the colony,
thus reinforcing the trail. As a result, the trail to the closest food source is the
6

strongest, thus without any central supervision the ant colony has located the closest
food source, thus optimizing their search.
Also, there is the concept of safety in numbers. A fish traveling in a school is much
safer from predators that it would be traveling alone. When attacked, the school can
split to evade the predator. Also, many predators will attack a solitary fish, but will
be wary of large school of fish that may retaliate en masse.
Finally, traveling as a group maximizes the distance able to be traveled. For
example, the V-shape formation observed in flocking birds is a functional formation
where the flight of the lead bird reduces the drag on the rest of the flock, thus enabling
the flock to travel farther as a whole. Table 2.1 provides other examples of swarming,
including some of the other benefits of swarming for both the individual and the
collective.
As seen through these examples of swarming in nature, evolution and natural
selection have tuned the behaviors of these swarms, enabling seemingly globally di-
rected behaviors to emerge solely through the interactions of the individuals in the
collective. Thus, these swarms are self-organizing systems where “the whole is greater
than the sum of the parts.” In order to further study the application of swarm intelli-
gence as a problem solving technique, a formal definition of what is meant by swarm
intelligence must first be established.
Swarm Behavior Individual Benefit Collective Benefit
Birds Flocking Reduces drag Maximize flock travel
Fish Schooling Enhances foraging Defense from predators
Slime Mold Aggregating Survive famines Find food quicker
Ants Foraging Enhances foraging Finding food faster
Termites Nest building Shelter Climate-controlled incubator
Table 2.1: Examples of swarm intelligence that can be found throughout nature.
7

2.2 Definition of Swarm Intelligence
In order to establish a concrete definition for swarm intelligence, a working definition
of agent must be established. For this work, an agent is defined as an entity that is
able to sense and affect its environment directly or indirectly. A swarm is then defined
as a collection of interacting agents. Finally, an environment is a substrate that
facilitates the functionalities of an agent through both observable and unobservable
properties. Thus, an environment provides a context for the agent and its abilities.
For example, an ocean is an environment to a fish in that it enables the fish to swim,
but factors such as viscosity affect the ability of the fish to swim.
With these definitions in place, a definition for swarm intelligence can be con-
structed. In order to construct this definition, one must consider the definitions
already existing in the field. As noted by Bonabeau, the term “swarm intelligence”
was first used by Beni, Hackwood, and Wang in regards to the self-organizing behav-
iors of cellular robotic systems [6]. This interpretation of swarm intelligence focuses
on unintelligent agents with limited processing capabilities, but possessing behaviors
that collectively are intelligent. Bonabeau [7] extends the definition of swarm intel-
ligence to include “any attempt to design algorithms or distributed problem-solving
devices inspired by the collective behavior of social insect colonies and other animal
societies”.
Clough sees swarm intelligence as being related solely to a collection of simple au-
tonomous agents that depend on local sensing and reactive behaviors to emerge global
behaviors [12]. Quite similarly to Clough, Payman [2] defines swarm intelligence as
the property of a system whereby the collective behaviors of unsophisticated agents
interacting locally with their environment cause coherent functional global patterns
to emerge.
The definitions of swarm intelligence outlined here are only a few of the many
varied definitions available, but from these definitions several key similarities can be
8

extracted. First, a swarm relies on a large number of agents. Secondly, the power of
a swarm is derived from large numbers of direct and/or indirect agent interactions,
thus the need for a relatively large number of agents. It is through the interactions
of the agents that their individual behaviors are magnified, thus resulting in a global
level behavior for the entire swarm. Finally, the agents in a swarm tend to be thought
of as simple because the complex emergent behaviors of the swarm is beyond their
capabilities and comprehension.
Based on the previously discussed definitions of swarm intelligence, we will estab-
lish a new definition of swarm intelligence that will be used for the purposes of this
work. Thus, given a group of agents considered simple relative to the observer, we
define swarm intelligence as
a group of agents whose collective interactions magnify the effects of in-
dividual agent behaviors, resulting in the manifestation of swarm level
behaviors beyond the capability of a small subgroup of agents.
In accord with our definition, we identify several key properties required for the
emergent behaviors observed in swarms. Foremost are large numbers of agent in-
teractions. These interactions create feedback, both positive and negative, which in
turn affects an agent’s context. Coupled over time with changing contexts, the con-
tinued interactions among agents accumulate, creating a feedback effect larger than
that realized by any single agent. This exploitation of magnified feedback pathways
is the key to swarm intelligence. Additionally, the interactions between agents not
only affect the agents but also the environment. The modification of the environment
imbues the agents with an indirect means of communication known as stigmergy [20].
In addition, randomness plays a large role in the success of a swarm algorithm.
When properly applied, random behavior can significantly reduce the size and com-
plexity of an agent’s ruleset by reducing the number of conditions that must be
handled. For example, a random walk combined with pheromone dropping serves
9

as a much simpler and robust foraging algorithm as compared to a more complex
algorithm that would need to account for the many situations that an agent in the
swarm would encounter. Of course, this reduction in choice is coupled with the loss
of some precision and efficiency. Also, randomness is an excellent way to escape from
local extremum, with respect to the problem at hand, without having to resort to a
case-by-case enumerative method. Fortunately, the robust nature of a swarm com-
pensates on a global scale for the possible misbehavior of a single agent, thus the
adverse effects of random behavior are relatively small.
2.3 Swarm Intelligence as a Problem Solving
Technique
As seen through the mentioned examples of swarm intelligence, a swarm algorithm
is a balance of ability and numbers. For example, assume an arbitrary task to be
completed. On one end of the spectrum, a single complex agent capable of performing
the task must possess all the intelligence and skill required to complete the task. This
enables the agent to quickly and efficiently address the task, but at a cost of robustness
and flexibility. If any of the required components of the agent fails, the agent cannot
complete the task. Additionally, when the ability of the agent needs to be expanded,
the complexity of the agent will be increased, thus possibly causing unforeseen issues
and reducing the maintainability of the agent. Finally, if the task is dynamic, a single
agent may have a difficult time of keeping up with the changing environment.
At the other end of the spectrum, for the same task, assume that you are given a
non-homogeneous swarm of agents who collectively have the ability to accomplish the
task. The swarm contains agents with redundant abilities, thus some duplicate work
will be performed but with the benefit that a malfunctioning or damaged agent does
not preclude the accomplishment of the task. Practically speaking, the complexity
10

of each swarm agent is much less than that of the single centralized agent, thus
maintenance and replication is much easier. Finally, though swarms may not always
optimally perform small or static tasks, as the size and dynamicism of a problem
grow, a swarm can scale with the problem with the addition of new agents.
Swarm algorithms are most useful for problems that are amenable to an agent-
based decomposition, have a dynamic nature, and do not require time-limited or
optimal solutions. Many problems today are ideal candidates for swarm algorithms,
including traveling salesman problems, data fusion on distributed sensor networks,
network load balancing, and the coordination of large numbers of machines, e.g., cars
on the highway or automatons in a factory.
The concept is an enticing one: program a few simple agents to react to their lim-
ited environment, mechanically replicate the agents into a swarm, and gain the ability
to realize complex goals. The resulting system is ﬂexible, robust, and inexpensive.
Flexibility comes through the ability of a swarm to adapt behaviors to operational
contexts. The system is robust because it provides for the possibility that several
agents can be lost or “go rogue” while the swarm still achieves its goals. In addition,
the size and composition of the swarm can change over time in conjunction with
evolving scenario parameters. Furthermore, this approach can often be less expen-
sive because many simple system components consume fewer resources to build and
maintain than a single, centralized system. Table 2.2 summarizes the main beneﬁts
of the swarm intelligence approach.
2.4 Applications of Swarm Intelligence
Many real-world problems are amenable to a swarm intelligence approach. Specif-
ically, the areas of computer graphics, scheduling, and networking have leveraged
swarm methodologies to create scalable, robust, and elegant solutions to complex,
11

Robust
Swarms have the ability to recover gracefully from
a wide range of exceptional inputs and situations
in a given environment.
Distributed and Parallel
Swarms can simultaneously task individual agents
or groups of agents to perform different behaviors
in parallel.
Effort Magnification
The feedback generated through the interactions
of the agents with each other and the environment
magnify the actions of the individual agents.
Simple Agents
Each agent is programmed with a rulebase that is
simple relative to the complexity of the problem.
Scalability
An increase in problem size can be addressed by
adding more agents.
Table 2.2: Benefits of using a swarm intelligence as a problem solving technique.
challenging problems. This section presents a summary of some of these real-world
applications of swarm intelligence.
Computer Animation
Boids are simulated flocking creatures created by Craig Reynolds to model the co-
ordinated behaviors bird flocks and fish schools [45]. The basic flocking model consists
of three simple steering behaviors that describe how an individual boid maneuvers
based on the positions and velocities of its neighboring boids: separation, alignment,
and cohesion. The separation behavior steers a boid to avoid crowding its local flock-
mates. The alignment behavior steers a boid towards the average heading of its local
flockmates. The cohesion behavior steers a boid to move toward the average position
of its local flockmates. These three steering behaviors, using only information sensed
from other nearby boids, allowed Reynolds to produce realistic flocking behaviors.
The first commercial application of the boid model appeared in the movie Batman
Returns, in which flocks of bats and colonies of penguins were simulated [44].
12

While boids modeled the flocking behaviors of simple flying creatures, Massive,1
a 3D animation system for AI-driven characters, is able to generate lifelike crowd
scenes with thousands of individual “actors” using agent-based artificial life technol-
ogy. Agents in Massive have vision, hearing, and touch sensing capabilities, allowing
them to respond naturally to their environment. The intelligence for the agents is
a combination of fuzzy logic, reactive behaviors, and rule-based programming. Cur-
rently, Massive is the premier 3D animation system for generating crowd-related
visual effects for film and television. Most notably, Massive was used to create
many of the dramatic battle scenes in the Lord of the Rings trilogy [27].
Scheduling
One of the emergent behaviors that can arise through swarm intelligence is that
of task assignment. One of the better known examples of task assignment within an
insect society is with honey bees. For example, typically older bees forage for food,
but in times of famine, younger bees will also be recruited for foraging to increase the
probability of find a viable food source [7]. Using such a biological system as a model,
Michael Campos of Northwestern University devised a technique for scheduling paint
booths in a truck factory that paint trucks as they roll off the assembly line [8]. Each
paint booth is modeled as an artificial bee that specializes in painting one specific
color. The booths have the ability to change their color, but the process is costly and
time-consuming.
The basic rule used to manage the division of labor states that a specialized paint
booth will continue to paint its color unless it perceives an important need to switch
colors and incur the related penalties. For example, if an urgent job for white paint
occurs and the queues for the white paint booths are significant, a green paint booth
will switch to white to handle the current job. This has a two-fold benefit in that first
the urgent job is handled in an acceptable amount of time, and second, the addition
1
http://www.massivesoftware.com
13

of a new white paint booth can alleviate the long queues at the other white paint
booths.
The paint scheduling system has been used successfully, resulting in a lower num-
ber of color switches and a higher level of eﬃciency. Also, the method is responsive
to changes in demand and can easily cope with paint booth breakdowns.
Network Routing
Perhaps the best known, and most widely applied, swarm intelligence example
found in nature is that of the foraging capabilities of ants. Given multiple food
sources around a nest, through stigmergy and random walking, the ants will locate
the nearest food source (shortest path) without any global coordination. The ants
use pheromone trails to indicate to other ants the location of a food source; the
stronger the pheromone, the closer the source [7]. Through a simple gradient following
behavior, the shortest path to a food source emerges.
Di Caro and Dorigo [10] used biological-inspiration derived from ant foraging to
develop AntNet, an ant colony based algorithm in which sets of artiﬁcial ants move
over a graph that represents a data network. Each ant constructs a path from a
source to a destination, along the way collecting information about the total time
and the network load. This information is then shared with other ants in the local
area. Di Caro and Dorigo have shown that AntNet was able to outperform static and
adaptive vector-distance and link-state shortest path routing algorithms, especially
with respect to heavy network loads.
The success of swarm-inspired network routing algorithms such as AntNet have
encouraged many in the communications industry to explore ant colony approaches
to network optimization. For example, British Telecom has applied concepts from ant
foraging to the problem of load balancing and message routing in telecommunications
networks [1]. Their network model is populated by agents, which in this case are mod-
14

eled as artiﬁcial ants. The ants deposit pheromone on each node traversed on their
trip through the network, thus populating a routing table with current information.
Call routing is then decided based on these routing tables.
15

Chapter 3
Swarm Simulation Software
This chapter describes the framework and implementation of SWEEP, the SWarm
Experimentation and Evaluation Platform. The implementation of SWEEP discussed
in this chapter is an extension of the original SWEEP implementation created by
Palmer and Hantak [22].
3.1 Existing Swarm Simulation Software
Aside from SWEEP, there are a number of existing swarm simulation packages that
are currently available. Perhaps the best known is Swarm.1
Swarm, originally
developed at the Santa Fe Institute, provides an extensive library of software com-
ponents designed for multi-agent simulations of complex systems. Breve2
is a 3-D
simulation environment designed for multi-agent simulations and artificial life re-
search . Breve includes an OpenGL display engine that allows a user to explore
simulated worlds from different angles and also provides the ability to create movies
from simulations. Finally, RePast3
(“Recursive Porous Agent Simulation Toolkit”)
is an agent-based simulation toolkit designed specifically for social science applica-
1
http://www.swarm.org
2
http://www.spiderland.org/breve
3
http://repast.sourceforge.net
16

tions. RePast permits the systematic study of complex system behaviors through
controlled and replicable computational experiments. RePast has an extensive set
of libraries relating to topics such as networking, statistics, and modeling. Another
feature of RePast is the built-in adaptive tools, such as a genetic algorithm package,
that allow a simulation to tune itself. A more comprehensive comparison of available
swarm/multi-agent software packages was performed in [48].
In terms of functionality, each of these simulation packages, including SWEEP,
contain similar abilities. Each is implemented in a cross-platform manner, provides
the user with an extendable architecture, and is not limited to any one type of simu-
lation. Where SWEEP differs from Swarm, Breve, and RePast is in that SWEEP
attempts to separate the programming required for modeling from the development of
the simulation itself. Thus for example, the initial implementation of a new agent will
require the development of Java code, but the usage of that agent will not require
any programming. In the current implementation of SWEEP, this goal is realized
through the XML simulation specification file that allows a user to build complex
simulations without the need to write Java code.
3.2 Design Overview
SWEEP identifies four major software components for a swarm simulation: Simulation,
Agent, Environment, and Probe. A Simulation object is composed of at least one
Agent object, one or more Environment objects, and zero or more Probe objects.
A Simulation object is responsible for constructing the various simulation compo-
nents, establishing the manner in which the simulation components communicate
and interact, and managing the runtime aspects of the simulation. Associated with a
Simulation is one or more Agents. An Agent represents a logical individual within
a simulation, which in implementation can in fact be a collection (“swarm”) of other
17

Agents. The Environment components associated with a Simulation define the
space where Agents exist, providing a context for the sensing and actuation. Finally,
Probes define methods of extracting internal simulation data from, and inserting
external information into, a running simulation.
The overall design goal for SWEEP was the creation of a flexible, general-purpose
swarm algorithm testing and development platform targeted for both users and devel-
opers alike. The high-level overview of the SWEEP design can be seen in Figure 3.2.
In order to separate the logic of an Agent and the simulation of the Agent, the basic
unit of autonomy in SWEEP is defined as an Entity, being composed of an Agent
(“mind”) and an Avatar (“body”). An Agent is defined as an object having both
a State and a Controller. The State is used as input to the Controller to de-
termine the next Action to be performed by the Avatar associated with the Agent.
The Avatar consists of a Model that defines the way the Avatar interacts with the
Environment. After executing the Actions from an Agent, the Avatar updates the
State of the Agent to reflect the effects of the executed Actions. Finally, all SWEEP
components communicate through Connectors, which allow for flexible and probe-
able inter-component communication.
3.3 Simulation
A SWEEP simulation is built from an XML simulation specification file. Within the
specification file are six subsections, each defining a portion of the simulation. Using a
pseudo-XML formatting, Figure 3.3 lists and defines the six major subsections found
in a SWEEP simulation specification.
Each XML subsection is passed to a Builder [16], which may be explicitly spec-
ified, to parse its own section. The preferred parsing method is to recursively pass
well-defined subsections to Builder objects that know how to handle that specific
18

Figure 3.1: High-level overview of the SWEEP architecture showing the relation-
ship between the major components.
data. This design guideline is used throughout the implementation of SWEEP. The
SimulationBuilder receives the simulation section of the specification file, and then
breaks that section into the six main sections. The SimulationBuilder then passes
each subsection to the appropriate Builder associated with that section. This pars-
ing strategy allows each section to define a custom syntax, which in turn provides for
more flexible and usable component architectures by separating the data format from
the data itself. For example, a Controller that uses a proprietary state machine
library can use either an XML state machine encoding or a custom state machine
encoding without requiring any modifications to the proprietary state machine code.
The default Simulation implements a sequential update strategy. First, the
Environment is updated to ensure that any Agent decisions are based on current
information. Then the Entities are updated by first updating the Agent that de-
termines which Actions are to be performed by the Avatar. Then the Avatar is
updated, executing commands from the Agent, resulting in an updated State for the
19

<simulation>
<main>
Defines how the specified components should
be connected together to create the simulation
</main>
<agent>
Describes how the controller and the avatar interact,
defines state variables that will be needed
</agent>
<controller >
Specifies the logic for an agent
</controller>
<model>
Defines the platform to be simulated, used by the avatar
</model>
<environment>
Setup of the context in which the agents interact
</environment>
<probes>
Specification of data extraction and insertion
method, can be used for interactive GUIs
</probes>
</simulation>
Figure 3.2: Shown here is a pseudo specification file for a SWEEP simulation.
The SWEEP simulation specification defines six major subsections. Each
subsection defines characteristics for different aspects of the simulation. The
file uses an XML format. Appendix A presents a complete SWEEP XML
specification file.
next Agent update. Finally, any defined Probes are executed. It is important to note
that SWEEP is not wedded to a sequential update strategy. Any update model, such
as a fully threaded or a distributed update model, can be implemented and used with
SWEEP.
3.4 Agent
An Agent has two main components, a State and a Controller. The State repre-
sents the current values of all variables that define the agent, and can be composed
20

of both manifest and latent variables [43]. A manifest variable is something that the
Agent has the ability to measure, whereas a latent variable is only indirectly observ-
able through the manifest variables (e.g.,, entropy). Many of the manifest variables
in a State object are values linked to Sensors in the Environment, thus providing
a mechanism for cleanly communicating Sensor information from the Environment
to the Controller. Any latent variables are inferred by the Agent using information
received directly from Sensors.
The Controller defines the logic by which the Agent is governed. Based on the
current State of the Agent, the Controller determines what Actions should be ex-
ecuted by the Avatar associated with the Agent. The default controller implemented
for SWEEP is a finite state machine [33]. Finite state machines were chosen for their
balance of power and simplicity. A state machine uses sensor information from an
Avatar as input, and Actions for the Avatar as output. Figure 3.3 illustrates a
trivial state machine that could be used by a SWEEP Agent. In the state machine,
transitions are fired based on the state of three binary sensors. When the sensors’
states match a desired condition, the specified action is performed and the state ma-
chine moves to the next state. For example, from state s-1, if all three sensors have
a value of true (1), then the action jmp will be executed and the state machine will
stay in state s-1.
In the simulation specification file, the state machine has two possible input for-
mats: state-based and transition-based. In the state-based version, each state is
defined by an XML section populated with the list of rules associated with that
state. In the transition-based version, the state machine section is populated by a
list of transition rules that indicated the source state associated with the rules. Fig-
ure 3.4 and Figure 3.5 show the state-based and transition-based formats of the state
machine defined in Figure 3.3.
21

Figure 3.3: A simple example of a finite state machine.
<state name="s-1">
<transition nextState="s-2" condition="101" action="esc"/>
<transition nextState="s-1" condition="111" action="jmp"/>
</state>
<state name="s-2">
<transition nextState="s-2" condition="000" action="add"/>
</state>
Figure 3.4: State-based XML format of the finite state machine in Figure 3.3.
<transition state="s-1" nextState="s-2" condition="101" action="esc"/>
<transition state="s-1" nextState="s-1" condition="111" action="jmp"/>
<transition state="s-2" nextState="s-2" condition="000" action="add"/>
Figure 3.5: Transition-based XML format of the finite state machine in Figure 3.3.
22

3.5 Avatar
The Avatar is the part of the Entity that extends into the Environment. The purpose
of the Avatar is two-fold. First, by separating the modeling of an Agent into an
Avatar, it is easier to keep free the logic of the swarm algorithm from dependencies on
environmental implementation issues. Secondly, the information abstraction ability
of an Avatar allows an Agent to seamlessly interact with either a computer-model
of the target platform, or an actual hardware platform whose sensors and actuators
are networked into the simulation. This allows earlier hardware testing of swarm
algorithms by mixing real and virtual platforms within the same scenario.
Every Avatar has an associated Model that dictates the characteristics and abil-
ities of the platform being simulated. For example, a UAV model will dictate a
minimum turning radius, maximum speed, and wing configuration. Sensors are the
most important aspect defined by the Model because Sensors provide decision infor-
mation to the Agent. The Model is also responsible for defining the Actions chosen
by an Agent. Finally, the Model and the Environment are responsible for determining
what, if any, direct methods of inter-agent communication are available.
3.6 Environment
The Environment defines the context in which the agents exist. The term “environ-
ment” is used in a very broad manner in order to create the most flexible definition for
the system. There are three core functionalities that the Environment is responsible
for:
1. Defining fundamental laws that Avatar objects must respect;
2. Presenting an information abstraction layer for Agent interaction;
3. Facilitating direct and indirect Agent interactions.
23

Perhaps the most important functionality of the Environment is the role it plays as
an information abstraction layer, which enables agents to “sense” information about
the environment without having to understand the implementation details of the
environment. For example, the concept of a neighborhood is functionally different
on a graph environment as opposed to a grid environment. However, to an agent
whose ruleset depends on neighbor information, the meaning of the data does not
need to change to accommodate different Environment implementations. Thus, the
information abstraction component of the Environment object facilitates the creation
of simulations where environments can be interchanged without a change to the logic
of the agent.
Additionally, the Environment is responsible for facilitating agent interactions.
One of the key components of any swarm algorithm is the large number of both di-
rect and indirect interactions between agents. Though some of these interactions may
occur externally between different agents, the majority of interactions will take place
within the context of the Environment, and thus will need to be mediated in order
to minimize the need for agents to understand the implementation of the environ-
ment. For example, when an agent uses a pheromone to stigmergically communicate
with other agents, an agent should not have to be concerned with how the chemical
gradient information is stored and processed, just that it exists. Thus, facilitating
agent interactions is a more specific way of presenting an agent-oriented information
abstraction layer.
Finally, the Environment defines a set of rules or laws that must be adhered
to by any Avatar associated with that particular Environment. Depending on the
Environment implementation, these laws can range from constants (like gravity or
the maximum network throughput) to constraints that must be satisfied or obeyed
(e.g., F=ma, or no more that three agents can be resident on a node at any one time).
It is through the definition and manipulation of these laws that environments with
24

varying levels of complexity and fidelity can be constructed and interchanged.
3.7 Probe
The ability to interact with a running simulation through Probes is one of the most
useful features of SWEEP. Probes in SWEEP serve two purposes: to extract in-
formation from a simulation, and to insert information into a simulation. Possible
uses for Probes include: generating animations of the simulation, storing statistical
information on a simulation, or producing a user interface to allow a user to examine
the effects of changing a parameter value during the execution of the simulation.
Due to the object models of modern programming languages, which restrict access
to memory based on coarse permission models, completely unrestricted access to data
for Probes is not entirely possible within the scope of a well-designed program. A
solution to this problem is the introduction of Connectors that serve as data conduits
between components in SWEEP. The Connector concept is inspired by a number of
design patterns, especially the Proxy [16] pattern and the Listener pattern as defined
by Java [3].
When a Connector is constructed, it is passed a reference to the two objects that
it connects. When one object wants to communicate with the other object, it actually
invokes methods on the Connector. The Connector then acts as a communications
abstraction layer. Any data passed through a Connector is made available to any
Probe. In a sense, Probes tap the communication lines in the same way one might
tap a phone line. Additionally, Probes are also able to delete or inject data into
a Connector. Thus, using Connectors, the full functionality of Probes is realized.
An example of how Probes might be used would be the construction of a diagnostic
interface for a simulation. The Probes provide the user interface with the current
status of each Agent in the simulation, allowing the user to monitor the progress of
25

each Agent individually or as a swarm. Additionally, the user can direct changes in
the environment, such as the availability of some resource, or even explicit changes
in the behavior of single or multiple agents.
26

Chapter 4
SWEEP Applications
This chapter presents three progressively more complex swarm demonstrations using
the SWEEP platform. The agent dispersion demonstration implements a sim-
ple algorithm that physically disperses a swarm to a predefined density. The task
assignment demonstration implements a decentralized task assignment algorithm
which is able to find “good solutions quickly”. Finally, chemical cloud tracking
is presented as a more comprehensive demonstration. In the scenario, a swarm of
UAVs (unmanned air vehicles) is tasked to search for and map a chemical cloud in a
bounded region. This demonstration includes UAV agents that respect the flight dy-
namics of a simple UAV, an environment with simple wind conditions and a chemical
cloud, and the use of SWEEP within a Monte Carlo analysis framework.
4.1 Dispersion
The goal of a swarm dispersion algorithm is to achieve positional configurations that
satisfy some user-defined criteria. Such criteria include distances from neighboring
agents, densities of neighboring agents, or the total area covered by the entire swarm.
Practical applications of dispersion algorithms for mobile agents include: conducting
a search of the Martian landscape using biologically-inspired tumbleweed robots [28],
27

forming an ad-hoc wireless network using autonomous UAVs or blimps [9], or as a
preparatory action for a more complex swarm algorithm. The key to the dispersion
algorithm is a measure of the quality of an agent’s position. The measure used in this
demonstration is a binary function, returning true if the agent’s position is good and
false if the position is bad, where good and bad are defined relative to the number of
neighbors for a given agent. Algorithm 1 provides a pseudo-code description of the
dispersion algorithm.
Algorithm 1 Simple dispersion
1: if Evaluate(pos) == Unsatisfied then
2: Move randomly
3: else
4: Do not move
5: end if
Figure 4.1: An example of the dispersion algorithm in action
The goal of each agent is to achieve and maintain a constant minimum and maxi-
mum distance from all neighboring agents. Figure 4.1 shows the basic situations that
will arise during the execution of a dispersion algorithm. As shown in the figure, an
agent can have unsatisfied positional constraints if they are either too close or too far
from other agents. An agent’s constraints will only be satisfied when none of their
neighbors are too far away or too close. To better illustrate the aggregate behavior
28

of the dispersion algorithm, a simulation with 125 agents on a 100 × 100 grid envi-
ronment was performed. Figure 4.2 shows the initial and final swarm configurations
of the simulation.
(a) Initial configuration before executing the
dispersion algorithm
(b) Final configuration after executing the dis-
persion algorithm
Figure 4.2: The initial and final configurations of a 125 agent swarm executing the dispersion
algorithm on a 100 × 100 region.
Table 4.1 is a summary of the simulation parameters used in implementing the
dispersion algorithm. Assume a swarm is randomly spread throughout a region, with
the goal of dispersing out to a certain density in order to provide a predetermined
level of area coverage. The goal of each agent is to achieve a position where no
neighboring agent is within 2 grid units, and there is at least one agent within 5 grid
units. Each agent is equipped with a distance sensor that can determine the distance
of neighboring agents who are within 6 grid units.
For each region size, 50 simulations were run using different random initial posi-
tions for each trial. The region boundaries provide an upper limit on the total area
covered by the swarm, thus region size can indirectly influence execution speed. In
the 50 × 50 case, on average the swarm disperses to a stable pattern within 300 time
steps. The 30 × 30 case takes substantially longer to emerge a stable pattern, usually
29

Number of Agents 100
Size of Region 20 × 20, 30 × 30, 50 × 50
Actions Move-RANDOM, Move-NONE
Sensors Neighbor-Distance
Dispersion Measure No agents closer than 2 grid units
and at least 1 agent within 5 grid
units
Table 4.1: Summary of dispersion simulation parameters
around 500–600 time steps due to the reduced amount of area with which to disperse.
Some simulations in the 30 × 30 case took over 1000 time steps to settle into a stable
pattern due to clusters of agents forming in a corner of the region, thus reducing the
number of useful moves available. Finally, the 20 × 20 case magnifies the clustering
problem seen in the 30 × 30 case as the agents themselves occupy one-quarter of the
available region space. A stable dispersion pattern, one where all agents simultane-
ously satisfy their positional constraints, is not found by the swarm in any of the
20 × 20 trials. As an experiment, several of the 20 × 20 simulations were allowed
to run for over 2000 time steps in an attempt to find a stable dispersion. Though
no stable dispersions are achieved, non-stabilizing dispersion patterns do begin to
emerge. A non-stabilizing dispersion can be thought of as the case where on average
the swarm is satisfying the aggregate positional constraints, but cannot satisfy each
agent’s individual constraints due to other factors, such as the size of the region.
4.2 Task Assignment
One of the many benefits of a decentralized multi-agent algorithm is the ability to
decompose a problem into parallelizable components. In order to realize the full
potential of a distributed multi-agent algorithm, a cooperative approach that co-
ordinates tasks in a non-overlapping manner becomes necessary. The Cooperative
Assignment of Swarm Tasks (CAST) auction [38] is a fully decentralized task assign-
30

ment algorithm based on synchronized random number generators that requires only
agent-local knowledge about all possible tasks. The CAST auction exhibits several
desirable features for a swarm including parallel computations, a limited number of
broadcast communications, workable solutions with very short computations, and ad-
ditional solution reﬁnement with increased computation time. In addition, the CAST
auction guarantees that with enough time the optimal solution will be found.
The basic assumptions of the CAST auction algorithm are minimal. First, assume
that each agent has a random number generator that can be synchronized across the
swarm. This can be done by broadcasting a common seed or having a common list
of seeds ready for use. The CAST auction also requires that all agents can perform
a broadcast communication that all other members of the swarm can receive. In
addition, assume that all agents either have or can compute their cost for each task.
The basic principle of the CAST auction is a time-limited, probabilistic search
of the full permutation space that yields good solutions with comparatively small
amounts of computation and communication. The steps of this approach are as
follows:
1. All agents compute their cost tables for the task independently.
2. Agents sort their choice lists in descending order of task cost.
3. Each agent internally computes the bid order for the current round of bidding.
4. The current bidder selects their best choice task and broadcasts the selected
task and associated.
5. On receiving a broadcasted selection, the selected task is removed from the task
pool and the task cost is added to the accumulated cost for this round.
6. If the end of a round is not reach, goto 4, else 7.
31

7. Repeat the auction with a diﬀerent bid order as needed. If the cost is less, the
new task mapping is adopted.
Table 4.2 is a summary of the simulation parameters used in implementing the
CAST auction algorithm. Each agent is capable of two actions, Receive-Bid which
listens for a bid broadcast, and Bid which selects the best task available with respect
to the bidding agent’s preferences and broadcasts that choice. Each agent is able to
determine if it is their turn to bid through the binary sensor Is-Turn-To-Bid which
returns true if it is the agent’s turn to bid and false otherwise.
Number of Agents 10, 50
Number of Tasks 10, 50
Number of Rounds 1, 10, 50, 100
Actions Bid-on-Task
Task Evaluation Criteria Cartesian distance
Table 4.2: Summary of CAST Auction simulation parameters
Figure 4.3: Results from a CAST auction with 10 agents, 10 tasks, and costs in the range
of (0, 1). Notice that the CAST auction quickly ﬁnds a near-optimal solution.
32

In order to compare results to those found in [38], assignments with 10 agents
and 10 tasks are simulated. For each instance of the problem, the position of both
the agents and the tasks were randomized. Figure 4.3 shows the results from one run
of the CAST auction with 10 agents, 10 tasks, and each task having a cost in the
range of (0, 1). The optimal assignment, as computed through brute force, has a total
cost of 0.6423. In round 9, the CAST auction finds an assignment with a total cost
of 0.6557, with successive rounds of auctioning producing no further progress. The
results of the CAST auction implementation are consistent with the results found in
[38], with near-optimal solutions (within 90%) being found within 10 bidding rounds.
4.3 Chemical Cloud Detection with UAV Swarms
In order to further test the simulation capabilities of SWEEP, a real-world problem
was selected for simulation and modeling within SWEEP. The scenario chosen in-
volves unmanned air vehicles attempting to coordinate in order to detect a chemical
cloud.
4.3.1 Introduction
Swarm algorithms can coordinate UAVs in situations that require a level of flexi-
bility and robustness unattainable by a single, more complex air vehicle controller.
Additionally, achieving effective use of UAVs requires more than just controlling a
single vehicle; the larger problem, with a potentially greater payoff, resides in es-
tablishing coordinated behavior of many such vehicles [12]. While much work has
been done applying emergent behavior and swarms to a wide variety of problems
[37, 8, 32, 5, 35, 19], there has been a deficiency in simulating realistic air vehicles
within swarms. Air vehicles have been modeled as particles [51], as objects that can
arbitrarily move within a hexagonal grid [42] or as detailed individual representations
33

of actual aircraft [25]. This simulation considers both issues: swarms of air vehicles
that respect the physical realities of their flight envelopes. The model of the flight
envelope is both general enough and realistic enough to apply to a wide variety of
UAVs. Also, an equivalent of randomized motion within motion constraints is defined
and is used to build simulations of UAV missions that exhibit emergent behavior.
The main simulation scenario used in the development and evaluation of UAV
swarm control strategies was that of a chemical cloud threat, as shown in Figure 4.4.
In this scenario, a UAV swarm is assigned a region to patrol, with the objective being
to determine if a chemical cloud is present. The swarm is deployed from a tower-like
structure, which could include the roof of a tall building. Once deployed, the UAV
swarm executes various distributed search algorithms. If a cloud is detected, many
actions can be taken, such as attempting to map the size and density of the cloud.
Also, if a UAV does find the cloud, and subsequently becomes contaminated, it must
not return to the home base, but to a predefined decontamination center.
(a) Deployment (b) Detection
(c) Mapping
Figure 4.4: The different stages of a chemical cloud detection mission: deployment, detec-
tion, and mapping.
34

4.3.2 System Overview
In order to aid the development and simulation of emergent behavior strategies for
UAVs, an optimal path planner that respects the UAV flight envelope is used. The
optimal path planner uses simple geometric operations to construct the optimal flight
path between two points for a UAV traveling at a constant speed. Given the destina-
tion and desired final heading, the optimal path planner constructs a control history
for the UAV [53].
Postulated UAV Configuration Swarm intelligence algorithms focus primarily
on emerging complex behaviors from simple agents. With this in mind, the postulated
UAVs are as minimalist as possible, thus allowing the developed strategies to be
applicable over a broad range of UAVs. The developed algorithms can be easily built
upon to leverage the capabilities of more advanced UAVs.
The UAVs are assumed to have a constant velocity of 40 knots. Computing
capabilities will be comparable to a 66 MHz computer. Some form of inter-agent
communication is assumed to be present, with no bias towards global communication
capabilities. The UAVs will have modular sensor arrays that can accommodate spe-
cific mission requirements. The battery or fuel cell for the UAVs should provide at
least one hour of operational time and be able to be recharged while detached from
the craft.
In this 2D model (Figure 4.5), the UAV is a point mass with its state specified
by (x, y, θ), where (x, y) is the instantaneous position and θ is the heading angle. In
addition, ω, which is the time rate of change of heading angle, is a controllable input.
It is assumed that the input ω is bounded by [−Ω, Ω]. Such bounds for the control
input are due to the limits on the flight envelope of UAV turning rate and speed. As
per the postulated UAV configuration, a constant velocity vC is assumed.
35




˙x = vC cos(θ)
˙y = vC sin(θ)
˙θ = ω
Figure 4.5: Postulated UAV flight model
4.3.3 Search Algorithms
UAV swarms are well-suited for tasks involving reconnaissance due to their inherent
parallelism and highly robust nature. A monolithic air reconnaissance platform can
only observe a single area at a time, and no information can be obtained if the aircraft
is disabled. These restrictions do not apply to UAV swarms. The parallel nature of a
UAV swarm can allow multiple areas to be observed by a single swarm, but also allows
for continued reconnaissance even if multiple members of the swarm are disabled.
Restrictions
The largest obstacle encountered when designing search strategies for UAV swarms
is managing the physical restrictions of the UAV. There are four main restrictions
when it comes to UAV performance: flight envelope, radar and communication, on-
board processing power, and fuel supply.
The flight envelope of a UAV is very restrictive. The cruising speed and turning
radius of the UAV are such that even the most basic search strategies need to account
for these factors. For example, if the objective is to locate or track a fast moving
object, the total area being simultaneously searched may have to be reduced in order
to improve the overall accuracy of the swarm due to the low cruising speed of the
individual UAVs.
Typically, in reducing the power, size, and cost of a UAV, the communication and
radar capabilities of the UAV are comparably diminished. These restrictions greatly
impact emergent behavior search strategies attempting to magnify the capabilities
of the swarm. For example, a swarm of UAVs with low-resolution sensor footprints
36

could locate an object smaller than their sensor footprint by merging their individual
sensor readings while still only requiring minimal communication overhead [12].
Search strategies for UAV swarms must expect limited on-board computation
capabilities, thus complex calculations will be either too slow or impossible to perform
in a real-time environment. Therefore, highly computational search strategies will not
be very effective.
Perhaps the most severe restriction is battery life. If a UAV can only be oper-
ational for one hour before its power supply is completely drained, then one knows
how sufficiently efficient and reliable a search strategy must be.
Randomized Search
Randomized searching is the most basic search strategy capable of being implemented
on a UAV swarm. Each UAV chooses a random heading and a random amount of
time, and then proceeds to fly in that direction for that duration. The operational
requirements for randomized searching are minimal. No communication capabilities
need be present on-board the UAV. The randomized search is adaptable for any
type of flight envelope and sensor array. As can be expected from any distributed,
randomized algorithm, results can only be guaranteed in a probabilistic manner.
When the number of UAVs is low, emergent behavioral algorithms suffer and perform
no better than any other strategy.
Symmetric Sub-region Search
A symmetric sub-region search divides the search area into equally shaped and sized
regions, assigning one or more agents to each region. For example in Figure 4.6(a),
eight circular subregions are defined around the perimeter of a known safe-zone.
In this scenario, the goal is to be able to detect when a threat incurs through the
perimeter. In Figure 4.6(b), no information is known about the region, so it is equally
37

probable that a threat can occur anywhere. Thus, the entire region is subdivided into
equally sized units of which a subset will be assigned to a each UAV to patrol.
Symmetric sub-region searching is useful when there is little a priori information
about the target (i.e., size or location), and when there are a relatively large number of
agents as compared to the size of the region to be searched. For example, if there exists
the threat of a chemical cloud, a symmetric sub-region approach could prove eﬀective.
Since the only information known is the wind heading and the fact that a threat
may exist, symmetric sub-regions allow for a higher probability that a threat will be
detected due to all of the simultaneous searches. This search strategy is only eﬀective
when each UAV is assigned a search area proportional to its sensor capabilities and
physical characteristics, otherwise large, unexamined gaps will remain.
(a) Radially Symmetrical Search (b) Grid Symmetrical Search
Figure 4.6: Examples of symmetrical search algorithms for UAVs. In Figure 4.6(a), the
symmetric subregions form a defense perimeter around a central point. This strategy as-
sumes that the area within the perimeter is clear and that the threat is coming from outside
the perimeter. In Figure 4.6(b), it is assumed the threat can be anywhere, thus each UAV
is responsible for a set of subregions.
4.3.4 Cloud Detection and Mapping
UAV Restrictions
The chemical cloud scenario imposes many restrictions on UAV swarms [11]. Gen-
38

erally, UAVs are capable of flying at speeds much greater than the average wind
speed, thus causing a problem in cloud mapping because a UAV can easily out pace
the cloud. The clouds involved in this scenario generally range from 1/2 km to 3 km in
length. The cloud is relatively small as compared to the 10 km2
region that must be
patrolled. Since our swarms only consisted of 5 to 15 UAVs, detecting clouds of this
size reliably made this scenario a good driving problem. Battery life was the largest
restriction on the UAVs’ capabilities. The UAVs evaluated had 1/2 hour of operational
time, thus making it critical to manage flight time with respect to returning to base
or a decontamination center. Additionally, each UAV was equipped with a chemical
sensor that possessed a 10 second processing latency.
Various enhancements to the UAV control schemes and search patterns arose from
the restrictions placed upon the scenario. In order to improve the chances that a
UAV swarm will successfully detect a chemical cloud, different search strategies were
considered with the modification that UAVs would search in manners that cross-cut
the wind, i.e., flying perpendicular to the wind direction. For example, the symmetric
sub-region search would have each UAV search their sub-region using this kind of
cross-cutting1
pattern, shown in Figure 4.6(a). The random search strategy would
have each UAV fly, and then at a randomly chosen moment cross-cut the wind for
a time, then continue a random flight. An added bonus of cross-cutting the wind
is that the UAVs can easily adjust to changing wind conditions by simply changing
their cross-cutting vector.
Cloud Mapping
Once a chemical cloud is detected by the UAVs, its location and heading are known.
By switching to alternative behavioral rules, the swarm can begin to ascertain the
cloud’s dimensions and density. With all types of mapping algorithms, the data
collected must be adjusted with time to take into account for the movement and
1
Cross-cutting is defined as flying perpendicular to the wind direction.
39

diffusion of the cloud. Either this can be done at a central location, or if capable, each
UAV can manipulate the data itself. When considering algorithms for cloud mapping,
the type of sensors that the UAVs are equipped with must be taken into account.
Binary sensors, which deliver a chemical present/absent signal, were assumed. There
are two types of mapping strategies we considered: inside-outside and dispersion.
The inside-outside method is very straightforward, as illustrated in Figure 4.7(a).
If an agent is inside a cloud, it chooses a direction and flies until it is outside of the
cloud. Once outside of the cloud, the UAV chooses a point randomly offset from the
last intersection with the cloud, and then flies back in. The points where the agent
exits or enters the cloud, transition points, are then recorded. From this data, a
model of the cloud size can be extrapolated.
(a) Inside-outside mapping (b) Dispersion mapping
Figure 4.7: Examples of inside-outside and dispersion cloud mapping algorithms for UAVs
A dispersion type algorithm can very effectively be used to map chemical clouds.
Essentially, once a UAV detects the cloud, it broadcasts this information to all sur-
rounding UAVs. After re-broadcasting the received message, the UAVs converge to
the general area that the cloud was initially located and begin executing a dispersion
algorithm. The agents execute the algorithm with only one modification; the UAVs
continue to expand the dispersion area until a UAV is out of the cloud. Thus, the
boundary of the cloud is used as an additional constraint on the overall dispersion.
At this time, the UAVs outside the cloud begin moving back into the cloud, as seen
40

in Figure 4.7(b). Thus, the shape of the cloud emerges.
4.3.5 Results
Chemical Cloud Mapping
In order to demonstrate the capabilities of emergent behavioral strategies, a UAV
rule-base was constructed for mapping a chemical cloud. The scenario used 10 UAVs
in a bounded 10 km2
region. Each UAV had a 30 minute power supply, a maximum
speed of 40 knots, and a sensor processing delay of 10 seconds. The wind had a
constant northeast direction at 4 knots.
Initially, a single UAV was placed in close proximity to the cloud while the rest
of the UAVs were randomly distributed throughout the region, thus simulating an
end-game situation for cloud searching. Quickly, a UAV finds the chemical cloud and
broadcasts the location of the cloud. Upon receiving the broadcasts, the UAVs swarm
to the general region where the cloud was found and begin the mapping process. For
simulation purposes, the inside-outside mapping method, as previously described, was
implemented.
Figure 4.8 shows a plotting of the data the UAV swarm collected in approximately
15 minutes of flight time. The white polygon represents the actual cloud. The
concentrations represent a post-processing of the data collected by the UAVs. As can
be clearly seen, the UAV swarm was able to determine the shape of the chemical cloud
relatively quickly. The mapping of the cloud is an emergent behavior. In all three
mapping strategies, no agent has any knowledge about the cloud’s size, shape, or rate
of diffusion. They simply randomly modify their behavior (change flight direction)
whenever they detect an inside/outside transition. By recording the time and location
of these transitions, the UAVs create a data set that contains all information necessary
to accurately depict the cloud’s movement over time.
41

(a) Initial detection of the chemical cloud. (b) The cloud begins to slowly drift out of range.
(c) The cloud is reacquired.
Figure 4.8: Cloud mapping results for a swarm of 10 UAVs. The white polygon represents
the actual cloud. The concentrations represent a post-processing of the data collected by
the UAVs.
Chemical Cloud Searching Scenario
The scenario chosen for the trade-oﬀ study was the detection of a chemical cloud by a
UAV swarm. The purpose of this trade-oﬀ study is to empirically determine the best
size for a UAV swarm searching a bounded region for a hostile-released chemical cloud.
In this scenario, a UAV swarm is deployed from a central tower in a square region.
Since the main objective of the swarm is detection of chemical clouds, it is unknown
to the swarm whether there previously existed a chemical cloud in the region they
patrol. Once released, the UAV swarm executes a search strategy in order to detect
42

the presence of a chemical cloud. If the cloud is not found before the UAVs’ power
supplies are depleted, the swarm returns to the tower. When the chemical cloud
is observed, the UAV swarm can transition into multiple cloud-mapping behaviors,
which report sensor-based information such as cloud size, density, and composition.
The swarm is responsible for patrolling a bounded square region. The intended
target is a chemical cloud of unspecified size or type that moves with the wind. The
chemical cloud is capable of diffusion, thus there is a fleeting window of time in
which the cloud can be detected before the cloud has diffused completely. Also, the
cloud diffusion causes the size of the cloud to increase, thus improving the chances of
detection, while at the same time causing a decrease in cloud density.
The UAV swarm used a modified random search to detect the cloud. Since the
UAVs are capable of detecting wind direction, a wind cross-cutting addition was made
to the randomized search. When en route to a randomly chosen destination, the UAV
can decide to randomly cross-cut the wind, thus increasing the chances of finding the
cloud. The UAV will cross-cut the wind for a random amount of time, then resume
the randomized search.
Though there are more structured search algorithms, the nature of the scenario
lends itself to the randomized search. As Clough states in [12], “random searches
are optimal given no a priori information.” Consider the case where the wind speed
is 0 m/s, thus the cloud does not move and has minimal dispersion. In this case,
more structured search algorithms will outperform the randomized search. Since the
cloud does not move, a previously searched location need not be searched again.
Thus, information does not become stale, and the swarm can systematically search
the region with confidence. In more reasonable cases, the wind is driving a cloud
along a path, and since the UAV swarm has no a priori knowledge about the cloud’s
location, information is only fresh for a wind-speed dependent amount of time.
We examined cases of the chemical cloud scenario with UAV swarms of sizes 5
43

through 15 in a region that measured 10,000 m2
. Cloud size varied from 1 km to 3
km in length, and 1 km to 1.5 km in width. The size of the cloud was regulated using
diffusion and decay parameters. The cloud was simulated as if there existed a single
moving source of the chemical (e.g., a crop duster).
The cloud lengths, beginning at 1 km, were incremented by 0.1 km. For each
cloud length, 100 simulations were executed for each discrete swarm size, starting at
5 and incrementing up to 15, for a total of 30,000 simulations. The simulations ran
until either the cloud was detected or the swarm ran out of fuel. The time to cloud
detection was recorded and presented in Figures 4.9–4.12.
As the figures indicate, there is a performance increase when the number of agents
is increased for the same-sized cloud. Figure 4.9 represents the normalized amount
of time taken by a UAV swarm to locate a chemical cloud. The search times were
normalized against the total amount of time with which the UAV swarm could have
searched. As expected, larger swarms were able to find similarly sized chemical clouds
faster than smaller sized swarms.
Figure 4.10 represents the percentage of times that the UAV swarm was able to
successfully detect the chemical cloud. An increase in the number of UAVs in the
swarm increases the swarm’s chances of finding the chemical cloud because proba-
bilistically speaking, more of the territory is being covered.
Figure 4.11 illustrates the average hit percentage of a UAV swarm of size n for
any sized cloud. Figure 4.12 represents the average amount of time taken by a UAV
swarm of size n to find any sized cloud. As can be seen, even with the maximum
number of agents, the chances of a UAV swarm finding a cloud of indeterminate size
is 83%. This performance rating may be acceptable for some situations, for example,
if the UAV swarm is used as an early detection system. As shown in Figure 4.11 and
Figure 4.12, there exists a linear improvement in the performance of the UAV swarm
with the addition of each agent.
44

Figure 4.9: This figure shows the normalized amount of time taken by a UAV swarm
to locate a chemical cloud. As expected, larger swarms were able to find similarly sized
chemical clouds faster than smaller sized swarms.
Figure 4.10: This figure shows the percentage of trials in which the UAV swarm was able
to successfully detect the chemical cloud. An increase in the number of UAVs in the swarm
increases the swarm’s chances of finding the chemical cloud because more of the territory
is being covered.
45

Figure 4.11: This figure shows the average hit percentage of a UAV swarm of size n for any
sized cloud. With the maximum number of agents, the chances of a UAV swarm finding a
cloud of indeterminate size is 83%.
Figure 4.12: This figure shows the average amount of time taken by a UAV swarm of size
n to find any sized cloud. Notice that there exists a linear improvement in the performance
of the UAV swarm with the addition of each agent.
46

Chapter 5
Evolutionary Generation of Swarm
Algorithms
5.1 Introduction
Swarms are complex systems whose behavior is difficult to determine a priori regard-
less of the predictive measures used. Although some recent progress has been made
on mathematical analysis of certain swarm algorithms [14, 15, 18, 21, 24, 26, 34, 36,
41, 49, 50, 52], the full simulation of a swarm algorithm is currently the best known
method of evaluating both the expected and emergent behaviors of the algorithm.
Thus, it is not surprising to find that the current state of the art in swarm algorithm
development is primarily relegated to a trial-and-error methodology. The realization
of desired high-level behaviors for a swarm without the need to define and program
the necessary lower-level behaviors would be an invaluable tool to both programmers
and non-programmers alike.
This chapter describes the development of ECS (Evolutionary Computing for
Swarms), an evolutionary computation engine designed to evolve swarm behavior
algorithms. The overall goal of ECS is the realization of an evolutionary comput-
47

ing architecture that is capable of automatically generating swarm algorithms that
exploit the inherent emergent properties of swarms with minimal user interaction.
5.2 System Overview
Figure 5.1 is a conceptual overview of ECS. The system is a composition of a component-
based evolutionary computing framework and the SWEEP swarm algorithm simula-
tor. From a high-level perspective, ECS operates similarly to a traditional evolution-
ary computing system [23]. An initial population of candidate solutions, represented
as finite state machines, are randomly generated. Each solution in the population is
then evaluated using SWEEP, resulting in a fitness score for each solution. The next
generation of the population is produced through the application of reproduction and
mutation operators on individual solutions selected by criteria such as relative fitness
or random selection. Finally, the entire evaluate-reproduce cycle is repeated until a
solution is found that satisfies the termination criteria. Algorithm 2 is a pseudo-code
representation of basic execution flow of ECS.
Figure 5.1: Conceptual overview of ECS (Evolutionary Computing for Swarms).
48

Algorithm 2 Pseudo-code for the basic functionality of ECS
Require: isValid(parameters.xml)
1: initialize(parameters.xml)
2: mutations = generateMutations()
3: sweep = createSWEEP()
4: population = generatePopulation()
5: gen ← 0
6: done ← false
7: while ¬done do
8: population = population ∪ createRandomSolution() {Helps prevent homoge-
nization}
9: for all m in mutations do
10: population = population ∪ m.selectAndMutate()
11: end for
12: for all x in population do
13: sweep.eval(x)
14: end for
15: if ∃x ∈ population such that isSolution(x) == true then
16: done ← true
17: end if
18: gen ← gen +1
19: if gen ≥ max generations then
20: done ← true
21: end if
22: end while
5.3 System Parameters
ECS has many configurable parameters that must be set in order to function. Ta-
ble 5.1 shows a sample set of parameters for evolving a dispersion algorithm. The
Objective exists for documentation purposes and is a short description of the behav-
ior that is being evolved. Maximum Generations specifies the upper limit on the
number of generations that ECS can execute before terminating execution. The Pop-
ulation Size specifies both the number of candidate solutions to initially create and
the number of solutions that will be allowed to survive between generations as se-
lected through elitism. Finally, the Number of Simulations indicates the number of
SWEEP simulations to average over when calculating fitness scores.
49

Each mutation is applied a specific number of times each generation. The resulting
mutated solutions are then rolled back into the population to compete for survival.
When specifying how to apply a mutation, a selection method and selection size
must be specified. Currently ECS supports random selection, elite selection (which
selects the top n solutions), and roulette selection (which selects solutions randomly
in proportion to their fitness relative to all other solutions) [4].
The specified Actions and Sensors are the primary components used to build
candidate solutions. Any valid SWEEP Action or Sensor is able to be included.
Currently, no validity checking is performed to ensure the compatibility of the Actions
and Sensors.
Finally, the parameters related to simulating the candidate solutions with SWEEP
are specified. The number and type of parameters will vary based on the simulations
needs of the fitness function used. In this case, the swarm size, environment size, and
maximum number of time steps are specified.
Parameters
Objective Dispersion
Max. Generations 500
Population Size 73
Number of Trials 2
Mutations
Change-Sensor-Value top 6 + 2 random
Change-Next-State top 6 + 2 random
Add-Transition top 6 + 2 random
Actions
move-random
move-none
Sensors
neighbor-left
neighbor-right
Simulation
Environment 50 × 50 grid
Maximum time 400
Table 5.1: An example set of parameters for evolving dispersion.
50

5.4 Solution Representation
In ECS, a solution is encoded as an extended Mealy state machine, where that state
machine encodes the program for a single agent in a homogeneous swarm. Instead
of using input symbols to trigger the firing of transition, the firing of each transition
is decided by a boolean condition (e.g., on-object==true). The boolean condition
compares a target sensor value with an actual sensor value. The condition is evalu-
ated to true when the actual sensor condition matches the target sensor condition,
resulting in the firing of the transition. When a transition fires, an action associated
with the agent is triggered, and the state machine goes to the next state as dictated
by the triggered transition.
There are many reasons why the state machine representation is the native rep-
resentation for candidate solutions. First, the state machine representation is simple
but yet capable of expressing complex logic. Also, the state machine representation is
inherently graph-based, thus previous work involving evolutionary computing using
graphs can be leveraged, especially with respect to defining evolutionary operators [4].
Finally, the state machine encoding is particularly enticing as its structure is robust
to random modifications,
Within the evolutionary computing framework, the candidate solutions are stored
as XML objects representing valid SWEEP controller structures. The benefit of this
representation is two-fold. First, the need for complex encoding/decoding of solutions
at evaluation time is eliminating as a simple serialization of the object results in a valid
XML file in the SWEEP state machine format. And second, mutation implementation
is simplified as genetic operators manipulate the solution directly by using XML
libraries, thus eliminating the possibility of accidentally creating malformed XML
content.
A small section of an XML encoded state machine is shown in Figure 5.2. Each
<state> contains at least one <transition>, and only one <default>. The <default>
51

specifies the action to execute and state to transition to when none of the defined
transitions fire. For this work, the default action is move-random and the default
next state is always the current state (e.g., the default next state for state A is A).
As seen in the figure, another benefit of the XML encoding is that extra information,
such as specific rule builder classes, can be embedded within the solution without
code modification or performance issues. Complete examples of XML-encoded state
machines can be found in Appendix B, Appendix C, and Appendix D.
<states>
<state name="e67e6a:10264a56b71:-7ff9">
<transition nextState="e67e6a:10264a56b71:-7fc5">
<rule logical="and">
<builder class="sweep.controller.SensorRuleBuilder"/>
<sensor name="holding-object" value="true"/>
</rule>
<action name="put-down-object"/>
</transition>
<default nextState="e67e6a:10264a56b71:-7ff9">
<action name="default-action"/>
</default>
</state>
...
</states>
Figure 5.2: A small section of a state machine solution in the native XML format.
5.5 Evolutionary Operators
The mutation operators defined for this work focus on the modification of the primary
components of the state machine representation: states, actions, and transitions.
Table 5.2 lists the mutations defined in this work.
Figure 5.3 graphically shows how each mutation operator affects a state machine.
All random selections performed by a mutation operator use a random number gen-
erator with a uniform distribution. The AddState mutation first creates a new state,
52

including randomly generated transitions to other states, and then connects that state
to the rest of the state machine by generating a transition from the state machine to
the newly created state. Given a selected state machine, the AddTransition muta-
tion first randomly selects two states from the state machine and creates a transition
between those two states, one as the start state and one as the next state. Then, a
random sensor condition and action are selected and assigned to the newly created
transition.
The mutations ChangeNextState, ChangeAction, and InvertSensor all start by
randomly selecting a transition t from the set of available transitions associated with
some selected state machine. The ChangeNextState mutation changes the next state
attribute of t to a randomly selected state within the state machine. The ChangeAc-
tion mutation changes the action attribute of t to a randomly selected action. Finally,
the InvertSensor mutation selects one of the sensor expressions associated with t and
inverts its condition (e.g., x == true becomes x == false).
The AddState and AddTransition mutations apply large changes to the struc-
ture of a candidate state machine solution, allowing for better exploration of the
solution space. Without these mutations, the number of states and transitions in
a solution would be fixed, thus purely relying on serendipity at the time of initial-
ization to randomly generate a state machine with a structure appropriate for the
problem. The mutations ChangeNextState, ChangeAction, and InvertSensor apply
finer-grained changes to state machine solutions, thus allowing state machines to be
debugged and tuned through many applications over several generations.
Mutation Description
AddState Add a new state and randomly connect to the state machine
AddTransition Add a new transition to a state
ChangeNextState Change the next-state attribute of a transition
InvertSensor Invert the sensor condition on a transition
ChangeAction Change the action attribute of a transition
Table 5.2: Mutations defined for a SWEEP state machine solution encoding
53

(a) Initial state machine (b) AddState mutation
(c) AddTransition mutation (d) ChangeNextState mutation
(e) InvertSensor mutation
Figure 5.3: Graphical illustrations of how the mutation operators function. Fig-
ure 5.3(a) shows the format of the initial state machine. Figures 5.3(b)–5.3(e)
successively show how each mutation affects the state machine. Note, Change-
Action is not shown, but functions similarly to ChangeNextState by randomly
selecting a transition then changing the associated action to a randomly selected
action.
5.6 Fitness Evaluation
The driving force in an evolutionary computation system is the fitness function be-
cause it imposes an ordering on the population of solutions, with ranking determined
by the aggregation of each solution’s ability to satisfy some set of metrics. Thus, the
fitness function acts as a differentiator, separating the adequate solutions from the
54

best solutions.
The idea is that the fitness function guides selection and mutation through a
selective pressure that manifests through a solution’s score relative to the score of
the entire population. Thus, a good fitness function filters a population on two lev-
els: separating “bad” solutions from “good” solutions, and by differentiating “good”
solutions from “better” solutions.
Each solution in the population is a state machine that encodes the program for
a single agent in a homogeneous swarm. Thus, simulation must be used in order
to evaluate the fitness of a solution. For this work, SWEEP is used to simulate a
homogeneous swarm where each agent is programmed with the logic encoded in the
state machine solution. The simulation is then run multiple times to remove any
biases introduced by the randomness of a swarm. The fitness function takes the
raw data generated by SWEEP, which is customizable and application specific, and
calculates a fitness score based on the evaluation metrics defined by the user. In
Chapter 6, the specific fitness metrics used for each scenario are discussed.
For the purpose of this work, we often normalize the fitness measure. Thus, with
normalization, the best achievable score is 1 and the worst score is 0. Using the
collection of objects as an example, say that out of 20 objects, solution s1 collected
7 objects and s2 collected 12. Now say that the fitness is based solely on the number
of objects collected. Thus, the normalized fitness score for s1 is 7/20 = 0.35 and for
s2 is 12/20 = 0.60. Since the goal is to maximize fitness, s2 is more fit than s1.
55

Chapter 6
Results of Evolving Swarm
Behaviors
The goal of this chapter is to develop several proof-of-concept scenarios showing how
evolutionary computing can be leveraged to design swarm algorithms. This chapter
presents the results of using the evolutionary computing system outlined in Chapter 5
to breed swarm algorithms for two basic swarm functionalities, swarm dispersion and
object manipulation. The goal of swarm dispersion is for every agent in the swarm
to satisfy some conditions of their positions resulting in a swarm of a certain density.
The swarm dispersion problem is relatively simple, but serves as a proof-of-concept
of the functionality of ECS. The second problem examined is object manipulation
where agent attempt to collect one type of object and destroy a different type of
object. As there are many different approaches that may be taken to address the
object manipulation problem, thus providing an adequate final scenario to test the
abilities of the ECS.
Each section in this chapter is broken down into three subsections: description,
implementation, and results. The first section gives a detailed overview of the problem
and the scenario. The second section outlines the agent configuration and parameters
56

used for the evolving the target behavior. The third section presents the results of
evolving the target swarm behavior.
6.1 Swarm Dispersion
6.1.1 Description
This section discusses the process of evolving a swarm behavior for agent dispersion.
Physical dispersion is a fundamental swarm behavior that has many applications
including establishing perimeters and performing searches. Additionally, dispersion
serves as a basis for more complex swarm behaviors as it provides a flexible method of
controlling the physical configurations of a swarm without the need to exert control
over each individual agent.
The goal of a dispersion algorithm is for a swarm is to achieve a physical con-
figuration that satisfies one or more constraints, such as average neighbor distance,
nearest-neighbor distances, or neighborhood densities. Since both an agent’s physical
dynamics and the dispersion criteria may not be static, the physical configurations
achieved by a dispersion algorithm can be either static or dynamic in nature.
6.1.2 Implementation
For this work, a swarm of agents is deployed in a bounded environment. In general,
the initial physical configuration of the swarm is arbitrary, so a random distribution
in a bounded subregion is used (see Section 4.1). The goal of the swarm is to expand
and/or contract to achieve a specific density, specified in terms of distances between
individual agents. More formally, an agent has satisfied the dispersion criteria if
all of their neighbors are at least dmin units away, but not more than dmax, where
dmin < dmax. Thus in this case, a swarm achieves a successful dispersion when all
agents meet the defined minimum and maximum neighbor distance criteria.
57

Figure 6.1: For an agent to satisfy the dispersion criteria, all of their neighbors are at least
dmin units away, but not more than dmax, where dmin < dmax.
The environment for the simulation is a bounded grid-based environment. The
agents possess limited mobility and sensing capabilities. Each agent can move in
any of the four cardinal directions at a uniform speed. Additionally, an agent is also
capable of moving in a random cardinal direction or not moving at all. The agent
behaviors related to movement are move-up, move-down, move-left, move-right, move-
random, and move-none.
Each agent has the ability to sense the presence of agents in its near vicinity. The
neighbor sensors are deﬁned for the four regions around the agent (Figure 6.1): above,
below, left, and right. Hence, the agent sensors available are neighbor-above, neighbor-
below, neighbor-left, and neighbor-right. Encoded in each sensor is the information
related to the dispersion criteria, i.e., dmin and dmax. Each sensor returns true only
when there are agents in the region that are violating the dispersion criteria of the
sensing agent. Thus, the goal for each individual agent is to ﬁnd a state where all of
their sensors report false.
58

The raw fitness of a solution is measured by the total number of agents violating
the dispersion criteria with respect to another agent. For example, if four agents are
arranged in a square, the fitness score would be 12 because each agent is too close to
the other three agents, thus violating the dispersion criteria for those three agents. A
worst-case scenario is where every agent is violating the dispersion criteria of every
other agent. If n is the total number of agents, this scenario establishes a maximum
error, MAX ERROR = n ∗ (n − 1). So, for the scenario examined here with 100
agents, MAX ERROR = 100∗99 = 9900. MAX ERROR is then used to normalize
the fitness score of each solution. Since the fitness function is also an error function,
the goal of this scenario is to minimize the error to 0.
The evolved solutions are scored using scenarios with randomized initial agent
positions. Table 6.1 shows the parameters used for evolving a swarm algorithm to
address the dispersion problem. All randomized operations (i.e., solution generation,
mutations, . . . ) use a random number generator with a uniform distribution. Ran-
domly generated solutions have exactly one state and one transition. All transitions
are generated by randomly assigning a start state and a next-state from the solution’s
set of states. The sensor condition and action associated with the transition are then
randomly selected from the solutions’s set of sensor conditions and actions.
Each of the five mutations are applied to eight solutions (the best 6 plus two
selected from random) per generation. Mutations are applied to a copy of the selected
solution, the copy is evaluated using SWEEP, and is then rolled back into the main
population. Thus, after all mutations are complete, the effective population size will
be 72 (the original 32 plus 40 mutated). Additionally, a single randomly generated
solution is introduced into the population at each generation.
Each solution is evaluated using SWEEP. Fitness data from each solution is col-
lected from SWEEP over 5 different trials. The data from all the trials is then
combined to give an accumulated set of raw fitness data.
59

Parameters
Objective Dispersion
Population Size 32
Number of Trials 5
Mutations
Change-Sensor-Value top 6 + 2 random
Change-Action-Value top 6 + 2 random
Change-Next-State top 6 + 2 random
Add-State top 6 + 2 random
Add-Transition top 6 + 2 random
Actions
move-up move-down
move-left move-right
move-random move-none
Sensors
neighbor-above neighbor-below
neighbor-left neighbor-right
Simulation
Maximum time 400
Table 6.1: Parameters for evolving dispersion
6.1.3 Results
In all of the evolutionary runs, a general fitness trend is observed. Initial generations
produce very poor results, with most candidate solutions scoring near the very bottom
of the scale. Gradual progress is then made for a short period, then a dramatic jump
in fitness occurs. The process of spiking and stabilizing continues one or more times
until a solution with the target fitness value is evolved. The characteristics of the
fitness over time is not typical of traditional evolutionary computing fitness trends,
which have a steady improvement as generations progress [4]. One large contributing
factor is the granularity of the fitness function. Traditional fitness functions produce
real-valued fitness measures, thus small improvements introduced through mutation
manifest themselves as slight improvements in fitness. In this case, since the fitness
function is integer based and thus discrete valued, the slight improvements introduced
60

through mutation may be overlooked. Thus, the fitness function is coarse-grained
and does not provide enough information to effectively drive the selective pressure
of evolution. Fortunately though in this case, the fitness function as defined does
produce enough information to drive evolution towards an acceptable solution.
A typical evolutionary run for dispersion is shown in Figure 6.2. In the first
few generations, fast progress is made as the initially random solutions are modified
through mutation. Then a period of no progress occurs, lasting for about 20 gener-
ations. However, after generation 29, there is a sharp decline in both the best and
average scores, indicating that one of the evolutionary operators made a change in one
of the solutions that resulted in an almost correct solution to the dispersion problem.
Yet another period of little progress occurs, this time lasting until a fully-functioning
dispersion algorithm is found in generation 111. The resulting evolved state machine
for dispersion is shown in Table 6.2.
61

Table 6.2: Evolved dispersion state machine
State Condition Action Next
State
A
! Neighbor-Above, ! Neighbor-Below, ! Neighbor-Left, ! Neighbor-Right no-op D
default move-random B
B
Neighbor-Above, ! Neighbor-Below, Neighbor-Left, Neighbor-Right move-random B
! Neighbor-Above, Neighbor-Below, ! Neighbor-Left, ! Neighbor-Right no-op F
! Neighbor-Above, Neighbor-Below, ! Neighbor-Left, ! Neighbor-Right move-random A
! Neighbor-Above, Neighbor-Below, Neighbor-Left, ! Neighbor-Right move-random B
Neighbor-Above, Neighbor-Below, Neighbor-Left, ! Neighbor-Right move-random B
default move-random A
C
Neighbor-Above, Neighbor-Below, Neighbor-Left, ! Neighbor-Right no-op B
! Neighbor-Above, ! Neighbor-Below, Neighbor-Left, Neighbor-Right no-op B
Neighbor-Above, ! Neighbor-Below, ! Neighbor-Left, Neighbor-Right move-random F
! Neighbor-Above, ! Neighbor-Below, ! Neighbor-Left, Neighbor-Right move-random A
Neighbor-Above, Neighbor-Below, ! Neighbor-Left, Neighbor-Right no-op B
Neighbor-Above, ! Neighbor-Below, ! Neighbor-Left, ! Neighbor-Right move-random B
default no-op C
D
! Neighbor-Above, Neighbor-Below, ! Neighbor-Left, Neighbor-Right move-random A
! Neighbor-Above, ! Neighbor-Below, ! Neighbor-Left, Neighbor-Right move-random D
default no-op C
E
! Neighbor-Above, Neighbor-Below, ! Neighbor-Left, Neighbor-Right move-random E
! Neighbor-Above, ! Neighbor-Below, Neighbor-Left, ! Neighbor-Right no-op B
default no-op B
F
! Neighbor-Above, Neighbor-Below, ! Neighbor-Left, ! Neighbor-Right move-random A
! Neighbor-Above, Neighbor-Below, ! Neighbor-Left, Neighbor-Right no-op B
Neighbor-Above, ! Neighbor-Below, ! Neighbor-Left, Neighbor-Right move-random A
Neighbor-Above, Neighbor-Below, ! Neighbor-Left, ! Neighbor-Right no-op D
default no-op E
62

0 20 40 60 80 100 120
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Generations
NormalizedFitness
Best and average fitness score for Dispersion
Best
Average
Figure 6.2: Best and average ﬁtness scores resulting from evolving a dispersion algorithm.
63

6.2 Object Manipulation
6.2.1 Description
In the object manipulation scenario, there are two types of objects scattered through-
out the environment, objects for collection and object for destruction. Thus, the
swarm has two simultaneous goals: (1) locate all collectible objects and return them
to a defined goal location and (2) locate and destroy all objects marked for destruc-
tion.
Object manipulation presents many interesting challenges for a swarm. Different
types of coordination are required to address each of the separate tasks, these being
collection and destruction. Object collection requires the development of a primarily
independent agent strategy. Though the agents can interact by communicating the
location of an object, the actual act of collecting an object can only be performed
by a single agent. On the other hand, object destruction requires the development
of a cooperative strategy because the destruction of an object is a multi-step process
and whenever an agent completes one of those steps they become disabled. From
a theoretical perspective, object manipulation is interesting because there is not a
trivial solution. There are many different approaches that may be taken to address
the object manipulation problem, and thus providing an adequate final scenario to
test the abilities of the evolutionary system developed in this work.
This section discusses the evolution of the basic object manipulation swarm algo-
rithm. A three phase approach will be taken for the evolution of an object manipu-
lation algorithm. First, the evolution of an object collection algorithm and an object
destruction algorithm will be addressed separately. Even though both are similar,
there are salient features of each problem that will need to be accounted for sepa-
rately. Then, leveraging the work from evolving the object collection and destruction
algorithms, an algorithm for the full object manipulation problem will be undertaken.
64

6.2.2 Implementation
This section will separately discuss the implementation details used for both the
object collection and object destruction scenarios. It is assumed that the object
manipulation scenario will use the combined implementations for object collection
and destruction.
Scenario Description
There are two types of objects randomly scattered throughout the environment, ob-
jects to be collected (type C) and objects to be destroyed (type D). The type C
objects are to be located and returned to a predisposed goal repository known to
all the agents. The type D objects require two separate “attacks” to be completely
destroyed. As a by-product of an agent attacking an object, that agent is disabled for
the duration of the simulation. Type D objects have three distinct states: untouched,
damaged, and destroyed. All type D objects originate in the untouched state, and
must become damaged before they are destroyed. When an agent first attacks an ob-
ject, the object transitions from the untouched to the damaged state. When another
agent attacks an object in the damaged state, the object transitions to the destroyed
state.
The agents in this scenario are free-moving and have the ability to randomly
wander. The initial physical configuration of the swarm is arbitrary, thus a random
distribution through the environment is assumed. An agent is able to sense and differ-
entiate between type C and D objects. Additionally, the ability to navigate directly
towards an object is incorporated to eliminate the need to evolve path planning ca-
pabilities, which is beyond the scope of this work. Finally, an agent has the ability
to distinctly broadcast to near-by neighbors the location of sensed objects. When an
agent receives a broadcast, the information is treated similarly to that of the object
proximity sensor, thus the broadcasted information is compatible with the navigation
65

behaviors.
Agent Behaviors
Table 6.3 and Table 6.4 provide a summary of the actions and sensors available as state
machine building blocks. For both cases, the default behavior when no transition fires
is move-random. Additionally, each agent is equipped with one sensor that indicates
when the agent is on an object, and one sensor that indicates when the agent is near
an object. With the implementation used in this work, when an agent is on an object,
the agent is also considered near an object.
The basic agent behaviors related to movement are move-up, move-down, move-
left, move-right, and move-random. Since the focus of this work is the evolution of
high-level algorithms, utility move-to behaviors are also provided to navigate an agent
to certain locations. Specifically, the move-to-goal behavior navigates the agent to
the known goal location, while move-to-object C and move-to-object D navigates an
agent toward the nearest type C or D object, respectively.
The pick-up and put-down behaviors are used to interact with type C objects. An
agent can only pick up a type C object when they are physically on top of the object,
and an object can only be put down when the agent is in the goal area. The first-
attack and second-attack behaviors are used to interact with type D objects. Each
type D object requires two separate “attacks” to be completely destroyed. Each
agent has the ability to execute either attack action. Additionally, a side-effect of
each attack is that upon a successful attack, the performing agent becomes disabled
for the duration of the simulation. Note, a minimum of 2n agents are required to
destroy n type D objects.
Finally, agents can communicate through the broadcast C and broadcast D be-
haviors, which broadcast to agents in a defined range the location of the nearest type
C or D object, respectively.
66

Agent Sensors
Each agent has a number of proximity sensors that provide them with information
about their environment. Two basic classes of sensors are defined, near and on.
A near sensor returns true when an agent is within a defined range of some ob-
ject, whereas an on sensor will only return true when an agent is physically on
some object. Note that with this implementation, when an agent is “on” an ob-
ject, they are also “near” that same object. Thus, a rule r1 using an on sensor as
input should be ordered before a rule r2 using a near sensor so as to ensure that
r1 is evaluated. In total, six sensors are defined: near-object C, near-object D, on-
object C, on-object D(untouched), on-object D(damaged), and on-goal. The two differ-
ent on sensors for type D objects allow the agent to differentiate between untouched
and partially destroyed objects, and hence differentiate when to use first-attack and
second-attack.
Behavior
Scenarios
Collection Destruction Manipulation
move-up x x x
move-down x x x
move-left x x x
move-right x x x
move-random x x x
pick-up x x
put-down x x
move-to-goal x x
broadcast C x x
move-to-object C x x
first-attack x x
second-attack x x
broadcast D x x
move-to-object D x x
Table 6.3: Object manipulation behavior primitives available for state machine construction.
67

Sensor
Scenarios
Collection Destruction Manipulation
near-object C x x
on-object C x x
holding-object C x x
on-goal x x
near-object D x x
on-object D(untouched) x x
on-object D(damaged) x x
Table 6.4: Object manipulation sensor primitives available for constructing state transitions.
Fitness Metrics
Table 6.5 describes the raw metrics that evaluate the performance of a candidate
solution with regards to the object collection and object destruction scenarios. The
metrics are formulated in terms of how much of a task is performed, thus the overall
goal is to maximize the fitness measures. Additionally, all metrics are evaluated on
the final timestep of the SWEEP simulation used to generate the raw fitness data.
Name Description
c1 number of objects put in the goal
c2 number of objects picked-up
d1 number of objects completely destroyed
d2 number of objects partially destrroyed
t normalized execution speed
Table 6.5: Basic fitness metrics for an object manipulation scenario.
The metric t measures the normalized execution speed of an evolved candidate
algorithm in terms of simulation timesteps. Given the maximum execution time as
400 timesteps (Table 6.7), and tactual as the measured length of the simulation in
timesteps, then t = 1 − tactual
400
. The metric t will have a value of 0 when a simulation
exceeds the maximum execution time. Thus, larger values of t indicate that the
evolved algorithm is executing faster.
Two metrics are defined to measure the fitness of a candidate object collection
algorithm. As described in Table 6.5, metric c2 measures the number of objects
68

picked-up, and metric c1 measures the number of object picked-up and returned to
the repository. For the individual solution fitness scores, the metrics are ordered in
decreasing importance: c1, c2, t.
Similarly, two metrics are defined to measure the fitness of a candidate object
destruction algorithm. As shown in Table 6.5, metric d2 measures the number of
partially destroyed objects, and metric d1 measures the number of completely de-
stroyed objects. For the individual solution fitness scores, the metrics are ordered in
decreasing importance: d1, d2, t.
As seen in the object collection and object destruction fitness metrics, the ordering
of the metrics also coincides with the sequential dependencies inherent in the scenario.
In the object manipulation scenario, the sequential dependency problem becomes
more pronounced as the sequential dependencies that exist for the individual object
collection and destruction subtasks are independent but equally weighted. Since c1
and d1 are equally weighted, the ordering of the two metrics skews the evolutionary
progress towards the metric given sequential preference. For example, when c1 is given
preference over d1, the solution pool tends to first evolve experts in object collection,
then the system has to modify through mutation an expert object collection algorithm
to accommodate for object destruction [29].
Since c1 and d1, and c2 and d2, are independent but equally weighted, a regular
ordering is not enough to adequately characterize the object manipulation scenario.
Thus, to address this issue, the raw fitness metrics are used to construct a set of
composite metrics that do not exhibit the sequentiality conflicts seen between the
raw metrics. Table 6.6 describes these composite metrics. The use of these composite
metrics enables the use of a radix sort to establish a fitness-based ordering. The
standard radix sort [13] moves from the least significant position to the most signifi-
cant position, resulting in a sorted list. For this work, the most significant metric is
m1 and proceeds to the least significant metric, m8. For boolean metrics (m1–m4),
69

true is deemed greater than false. Figure 6.3 shows a simple example of how radix
sorting would be applied to a small population of solutions (more fit solutions at the
top).
Unsorted Sorted
{ true 5 3 9 } { true 8 3 6 }
{ true 8 1 6 } { true 5 3 9 }
{ false 4 2 7 } { true 5 3 3 }
{ true 5 3 3 } { false 4 2 7 }
Figure 6.3: A simple example of radix sorting. Note that true is deemed greater than
false, and that the solutions are sorted from most to least fit.
Name/Priority Metric Description
m1 c1 ∧ d1 flag solutions fully performing both subgoals
m2 c2 ∧ d2 flag solutions partially performing both subgoals
m3 c1 ∨ d1 flag solutions fully performing either subgoal
m4 c2 ∨ d2 flag solutions partially performing either subgoal
m5 min(c1, d1) select the weakest of the two primary subgoals
m6 min(c2, d2) select the weakest of the two secondary subgoals
m7 max(c1, d1) select the strongest of the two primary subgoals
m8 max(c2, d2) select the strongest of the two secondary subgoals
m9 1 − tactual/tmax normalized execution speed
Table 6.6: Composite metrics used for the object manipulation scenario. The metrics are
created from raw fitness data in order to eliminate the conflicting sequential dependen-
cies that exist in using the raw fitness metrics. Note, the value of tmax is 400 timesteps
(Table 6.7), and tactual is the candidate solution’s execution speed in timesteps.
In order to create the correct ordering needed for the object manipulation scenario,
first a series of boolean flags are defined. These flags ensure that the solutions that
meet the flag criteria are ranked higher than those that do not. Thus, higher-level
logic can be incorporated into the radix scoring, such as the metrics m1 and m2
which encode the rule “solutions that act on both subgoals are better than solutions
that address only one subgoal.” Metric m3 ensures that a solution that completely
addresses one subgoal is ranked higher than a solution that only partially addresses
a subgoal, regardless of the raw metric values. Metric m4 ensures that solutions
that partially address any subgoal are ranked higher than solutions that do nothing.
70

Though m4 is not needed, it is included for completeness. Metrics m5 and m6 rank
solutions according to their weakest subgoals, focusing on c1/d1 first and then c2/d2.
These metrics are implemented in this fashion because a solution’s total fitness is
only as good as the weakest component. Similarly, metrics m7 and m8 rank solutions
according to their strongest subgoals, focusing on c1/d1 first and then c2/d2.
Table 6.7 summarizes the parameters used in the evolution of solutions for object
collection, destruction, and manipulation. All randomized operations (i.e., solution
generation, mutations, . . . ) use a random number generator with a uniform distribu-
tion. All randomly generated solutions are created with exactly one state s and one
transition t. All transitions are generated by randomly assigning a start state and
a next-state from the solution’s set of states. For randomly generated solutions, the
state s is both the start state and next-state for the transition t. The sensor condition
and action associated with a transition are randomly selected from the solutions’s set
of sensor conditions and actions, respectively.
Each of the five mutations are applied to eight solutions (the best 6 plus two
selected from random) per generation. Mutations are applied to a copy of the selected
solution, the copy is evaluated using SWEEP, and is then rolled back into the main
population. Thus, after all mutations are complete, the effective population size will
be 72 (the original 32 plus 40 mutated). Additionally, a single randomly generated
solution is introduced into the population at each generation.
Each solution is evaluated using SWEEP. Fitness data from each solution is col-
lected from SWEEP over 2 different trials. The data from all the trials is then
combined to give an accumulated set of raw fitness data which is then passed to the
fitness function.
71

Parameters
Objective Object Manipulation
Population Size 32
Number of Trials 2
Mutations
Change-Sensor-Value best 6 + 2 random
Change-Action-Value best 6 + 2 random
Change-Next-State best 6 + 2 random
Add-State best 6 + 2 random
Add-Transition best 6 + 2 random
Actions
move-random move-to-object C
move-to-goal move-to-object D
pick-up put-down
ﬁrst-attack second-attack
broadcast-object C broadcast-object D
Sensors
on-object C near-object C
on-goal holding-object C
on-object D(untouched) on-object D(damaged)
near-object D
Simulation
Maximum time (tmax) 400
Number objects type C 50
Number objects type D 30
Broadcast range 25
Sensing range 5
Table 6.7: Parameters for evolving object collection, destruction, and manipulation
72

6.2.3 Results
Viable solutions to each of the three scenarios outlined were successfully evolved. The
performance for the object destruction and object collection scenarios were similar,
with each finding a solution that completely addresses the secondary subgoal for the
scenario. Then, within 25 generations a complete solution is found, with a somewhat
time optimized solution being evolved within 20 generations after that. The object
manipulation scenario has a longer convergence time, taking at least 300 generations
to evolve a full solution, and up to 40 more generations to evolve a more time efficient
solution.
The object destruction and object collection scenarios each had similar average
convergence times, with object destruction being the slightly easier of the two sce-
narios. From a practical standpoint, the object destruction scenario is the easiest of
the three scenarios because the sequential subtasks are assigned to the swarm, not
an individual agent. Specifically unlike the object collection scenario where a single
agent is responsible for both picking up and depositing an object, an agent in the
object destruction scenario is only responsible for one “attack” on an object, relying
on another member of the swarm to complete the job. Thus, evolution of object
destruction behaviors is easier because the sequential dependencies on a single agent
are eliminated.
Separately looking at the average convergence times for the collection and de-
struction tasks in the object manipulation scenario (Figure 6.8 and Figure 6.10), the
same convergence rate is not observed as when the tasks are evolved separately (Fig-
ure 6.4 and Figure 6.6). This performance decrease is expected, as the collection
and destruction tasks are in direct competition for the same pool of fitness resources.
So, the evolution of both behaviors simultaneously increases the competition faced
by each solution at every generation. In the early implementations of the system,
this competition actually inhibited convergence because the the fitness function used
73

a more traditional approach of weighting individual metrics, which led to the de-
velopment of “expert” solutions [29]. In this case, an expert solution is one that is
able to completely satisfy one goal but not the other, such as collecting all type C
objects but not destroying any type D objects. These expert solutions dominated
the population, thus eventually they dominated reproduction, resulting in an almost
homogeneous population. Once the population was mostly homogenized, the system
then focused on using mutations to transform an expert solution into a solution that
addressed both goals. The homogenization identified a need for a method to balance
the evolution between multiple objectives, hence the introduction of the radix ranking
method. As seen in Figure 6.8 and Figure 6.9, the radix ranking is able to balance
the evolution towards both collection and destruction. The simultaneous progress
is made on each secondary subgoal, m6 and m8, and each primary subgoal, m5 and
m7, thus enabling the simultaneous evolution of a complete solution for the object
manipulation scenario.
Object Destruction
The average number of generations required to evolve a solution for object destruc-
tion is approximately 50 generations. As seen in Figure 6.4, within 2 generations
a solution is found that completely maximizes d2, and within 5 generations most of
the population is able to almost maximize d2, as shown by the mean fitness in Fig-
ure 6.5. As the population drives towards maximizing d2, slow progress is eventually
made towards the maximization of d1. Mutated solutions that begin to maximize d1
rise to the top of the fitness scale, thus being selected more often for reproduction.
Eventually in generation 14, a solution is produced that completely maximized d1
and d2. As the population converges towards a set of complete solutions, the last
metric to be addressed is the time t. It is interesting to note how the mean fitness
for the population decreases once a complete solution is found. This behavior can
74

be attributed to the mutations being applied during reproduction since at this point
most of the solutions in the population maximize d1 and d2, thus the probability that
a random mutation will be helpful is small.
A solution that completely maximizes both d1 and d2 is found in generation 14,
but approximately 5 more generations of evolution is able to produce a solution that
executes quickly. The solution evolved here, shown in Table 6.8, is able to completely
destroy all 30 objects in approximately 30 timesteps. Table 6.9 is a simplified version
of the evolved object destruction algorithm shown in Table 6.8. The simplifications
include the removal of unreachable states and non-firing transitions, and the removal
of actions that will have no effect.
Every agent executing the algorithm in Table 6.9 starts in state A. If the agent is
on any type D object, the appropriate attack will be executed and the agent will be
disabled for the duration of the simulation. If the agent is not on a type D object, the
agent transitions to state E, where they will randomly wander until they are within
sensing range of a type D object. Though unlikely here, if the agent stumbles upon
a partially destroyed type D object, they will attack it appropriately and become
disabled. What is more likely to happen that the agent will come in proximity to a
type D object, and since they are not on a partially destroyed type D object, they will
move towards the nearest type D object and transition from state E to C. Implicit
in the transitions from state E to state C is that the agent is near a type D object,
thus in state C the first transition fires, moving an agent towards a type D object and
transitioning back to state A, where this cycle repeats until all the type D objects
are destroyed.
The transition from state E to state C has the implied condition that the agent is
near a type D object, say X, but since there are multiple agents executing this same
algorithm, a race condition occurs. It is conceivable that in the transition from E to
C another agent completes the destruction X, which will probably cause the agent
75

that transitioned from E to C to execute the second transition in state C, sending
the agent to state J. In state J, the agent will randomly wander until it is sitting on
top of an untouched type D object, say Y , at which time it will wander off Y and
transition to state F. In the unlikely event that the agent wanders onto a partially
destroyed type D object, the second transition in state F will fire, completing the
destruction of a type D object and disabling the agent. More often though, the agent
will execute the first transition in state F, bringing the agent back on top of Y and
causing a transition to state H, where the agent will enter a do-nothing infinite loop
between states H and D. Only once another agent attacks Y will the agent be able
to break from the infinite loop. In the end though, even if the agent is broken free
from the H/D infinite loop, the agent is relegated to transitioning between states B,
D, H, and I which do not provide much help in accomplishing the task of object
destruction.
76

State Condition Action Next State
A
on-object D(untouched) ﬁrst-attack A
!on-object D(damaged) move-towards-object D E
near-object D second-attack F
B
!near-object D broadcast-object D H
!on-object D(untouched) move-towards-object D B
near-object D second-attack I
C
near-object D move-towards-object D A
on-object D(untouched) ﬁrst-attack H
!on-object D(damaged) move-random J
on-object D(damaged) move-random A
D
!on-object D(untouched) move-random B
!on-object D(damaged) second-attack H
!near-object D move-towards-object D I
E
!near-object D move-random E
on-object D(damaged) second-attack C
on-object D(untouched) move-random E
!on-object D(damaged) move-towards-object D C
F
!on-object D(damaged) move-towards-object D H
!on-object D(untouched) second-attack I
H
!on-object D(untouched) broadcast-object D H
on-object D(untouched) second-attack D
I
!near-object D second-attack H
near-object D second-attack B
J on-object D(untouched) move-random F
Table 6.8: Evolved object destruction state machine
77

A
on-object D(untouched) ﬁrst-attack -
!on-object D(damaged) move-to-object D E
near-object D second-attack -
B
!near-object D - H
!on-object D(untouched) move-to-object D B
near-object D second-attack I
C
near-object D move-to-object D A
!on-object D(damaged) move-random J
D
!on-object D(untouched) move-random B
!on-object D(damaged) - H
!near-object D - I
E
!near-object D move-random E
on-object D(damaged) second-attack -
on-object D(untouched) move-random E
!on-object D(damaged) move-to-object D C
F
!on-object D(damaged) move-to-object D H
!on-object D(untouched) second-attack -
G on-object D(damaged) second-attack I
H
!on-object D(untouched) broadcast-object D H
on-object D(untouched) - D
I
!near-object D - H
near-object D second-attack -
J on-object D(untouched) move-random F
Table 6.9: Simpliﬁed version of the evolved object destruction state machine
78

5 10 15 20 25 30 35 40 45 50
0
0.2
0.4
0.6
0.8
1
Best fitness for Object Destruction
Generation
NormalizedFitness
d1
d2
t
Figure 6.4: Progress of the best solution over 50 generations for object destruction is shown.
In generation 14, a solution that maximizes the fitness measures d1 and d2 is found. Around
generation 19, a more time-optimized solution to the object destruction problem is found.
5 10 15 20 25 30 35 40 45 50
0
0.2
0.4
0.6
0.8
1
Mean fitness for Object Destruction
Generation
NormalizedFitness
d1
d2
t
Figure 6.5: Mean fitness over time for object destruction is shown. Notice how the mean
fitness begins to decrease as soon as the best solution is found at generation 19. This is
attributed to the mutations being applied during reproduction since at this point most of
the solutions in the population maximize d1 and d2, thus the probability that a random
mutation will be helpful is small.
79

Object Collection
Over 8 separate evolutions, the average number of generations required to evolve a
solution for object collection is approximately 55 generations. As seen in Figure 6.6,
within 5 generation a solution is found that completely maximizes c2, and within
10 generations a majority of the population is able to almost completely maximize
c2, as shown by Figure 6.7. As the population drives towards maximizing c2, slow
progress is eventually made towards the maximization of c1. Mutated solutions that
begin to maximize c1 rise to the top of the fitness scale, thus being selected more
often for reproduction. Eventually in generation 21 a solution is produced that com-
pletely maximized c1 and c2. As the population converges towards a set of complete
solutions, the last metric to be addressed is the time t. The performance of the best
solution with respect to t up until generation 45 is quite varied, indicating that the
solutions are still relying on some form of serendipity to complete the task. As seen
in Figure 6.6, after generation 45, four marked periods of consistent performance oc-
cur. During these periods, a single solution dominates the population, thus this one
solution receives a majority of the mutations. Hence at this point, the optimization
of t has essentially become a pure random search, using the mutation operators to
explore the solution space around the best solution. The additional work required
to optimize t is due to the fact that agents do not receive credit for navigating to
the object repository unless they arrive there and successfully deposit the object they
are holding. Additionally, similar to what is seen in evolving object destruction, the
mean fitness for the population increases once a complete solution is found.
The solution evolved for object collection is shown in Table 6.10, and a simplified
version of the algorithm is shown in Table 6.11. The simplifications include the
removal of unreachable states and non-firing transitions, and the removal of actions
that will have no effect.
When executing the solution in Table 6.11, each agent starts in state A. No agent
80

is holding any type C objects at the beginning, so the second transitions in state A
cannot fire. Thus the first time through state A, the agent eventually will select the
first transition, attempt to pick up and object, and goto state E. When transitioning
from state A, the first two transitions in state E cannot fire, but either the third or
fourth will definitely be activated. These two branches will be discussed separately.
First, if the agent is not near any type C object, the third transition of state E
will fire, resulting in going to state B. As a consequence of the transition to state B
from E, the first two transitions in state B cannot fire. If the agent is not holding a
type C object, the agent will transition to state C and eventually end up back in state
A. If the agent is holding a type C object, then the third transition in state B fires,
navigating the agent towards the goal area and transitioning into state E. At this
point, if the agent is in the goal area (which is unlikely), the agent will transition from
state E to state G, an inescapable loop that will cause the agent to hold onto the type
C object forever. Now, in order for the agent to continue navigating towards the goal
area, there cannot be any nearby objects preventing the firing of the third transition
in state E. If there is an object nearby, the agent will navigate towards it, broadcast
its location, and wait to deliver its object until the other object is collected. After
cycling like this through states B and E until the agent is in state B and located
in the goal area, the first transition in state B fires, and assuming that there are
not type C objects around, the agent will not move and will transition to state A,
resulting in the completion of collecting one object. Unfortunately, from here the
agent transitions from A → F → E → G. Thus, after successfully collecting an
object, the agent becomes useless as it is stuck in the infinite loop at G.
The other branch point initially reached at state E happens when the fourth
transition fires because an agent is near an object. The agent will cycle through
states E → C → A → E, navigating towards the object until it is picked up in state
A. At this point, the agent is holding an object and is in state E, which is the same
81

situation previously discussed.
The evolved object collection algorithm does exhibit a form of emergent behavior.
Given the algorithm in Table 6.11 and a swarm of agents, the object collection task
is performed quickly and completely. But, when this algorithm is executed with only
one or a few agents, the swarm fails to complete the object collection. What happens
is that an agent picks up an object in state A, transitions to state E, and if the agent
is near any other objects, they navigate towards that object and broadcast its position
until another agent picks up the object. But in this case with one or few agents, there
is no one able to collect the object and break the loop. Thus, the agent(s) are locked
in a broadcasting loop and never complete the object collection task.
82

A
!on-goal pick-up E
holding-object C put-down F
on-object C move-to-goal G
B
on-goal move-to-object C A
near-object C broadcast-garbage D
on-object C move-random G
holding-object C move-to-goal E
!on-object C move-random C
C
!on-goal move-to-object C A
on-object C pick-up E
near-object C put-down B
D
!on-goal put-down F
!near-object C put-down B
on-object C move-random G
E
on-goal move-random G
on-object C put-down B
!near-object C move-to-goal B
!on-object C move-to-object C C
F
!holding-object C put-down E
holding-object C move-to-object C B
G !holding-object C move-to-object C G
Table 6.10: Evolved object collection algorithm
A
!on-goal pick-up E
holding-object C put-down F
B
on-goal move-to-object C A
near-object C broadcast-garbage D
holding-object C move-to-goal E
!on-object C move-random C
C
!on-goal move-to-object C A
near-object C put-down B
D
!on-goal - F
!near-object C put-down B
E
on-goal move-random G
on-object C - B
!near-object C move-to-goal B
!on-object C move-to-object C C
F
!holding-object C - E
holding-object C move-to-object C B
G !holding-object C move-to-object C G
Table 6.11: Simpliﬁed version of the evolved object collection algorithm
83

50 100 150 200 250 300 350
0
0.2
0.4
0.6
0.8
1
Best fitness for Object Collection
Generation
NormalizedFitness
c1
c2
t
Figure 6.6: Progress of the best solution over time for object collection is shown. In gen-
eration 21, a solution that maximizes the fitness measures c1 and c2 is found. Around
generation 150, a time-optimized solution to the object collection problem is found.
50 100 150 200 250 300 350
0
0.2
0.4
0.6
0.8
1
Mean fitness for Object Collection
Generation
NormalizedFitness
c1
c2
t
Figure 6.7: Mean mean fitness over 350 generations for object collection is shown. Notice
how the mean fitness begins to decrease as soon a the best solution is found at generation
150. This is attributed to the mutations being applied during reproduction since at this
point most of the solutions in the population maximize c1 and c2, thus the probability that
a random mutation will be helpful is small.
84

Object Manipulation
The final scenario presented to the system is the complete object manipulation sce-
nario involving the simultaneous collection and destruction of all the objects in the
environment. The average behavior of the evolution of an object manipulation solu-
tion is much less well defined than the behavior of either of the other scenarios. As
shown by Figure 6.8 and Figure 6.10, two different runs of object manipulation can
have very different behavior with respect to convergence. In Figure 6.8, almost 325
generations are required to evolve a solution that completely collects and destroys all
the objects in the environment, but only around 160 generations for the run shown
in Figure 6.10. Additionally, the best solution from the run shown in Figure 6.8
used 80% of the allotted runtime, whereas the best solution from the run shown in
Figure 6.10 uses only 50% of the allotted runtime.
Over 8 different trials, evolving a solution for object manipulation required ap-
proximately 300 generations to find a complete solution, and at least 100 additional
generations was required to begin time optimizing a solution. The convergence time
for the combined object collection and destruction scenarios is much longer than that
of the individual scenarios. This performance is expected as the multiple independent
objectives compete with each other for survival.
The balancing effect of the radix sorting can be seen in both Figure 6.9 and Fig-
ure 6.11, preventing one metric from dominating and skewing the mean performance
of the population towards that metric. This point is clearly illustrated through the
progress of the mean population performance shown in Figure 6.9. Metrics m6 and
m8, representing the secondary subtasks for object collection or destruction, and met-
rics m5 and m7, representing the completion of the object collection or destruction
task, do not deviate far from each other. Also notice the effect of the radix scoring
and how neither of the two main tasks (collection or destruction) take a commanding
lead over the other task. Additionally, unlike the object collection and destruction
85

evolutions, due to the larger search space the mean fitness does not begin to increase
after a good solution is found.
The performance of the radix scoring is observed in Figure 6.14; note how the
number of solutions that satisfy one of the four metrics m1, m2, m3, or m4 increases
with respect to the progress made by the best solution. This same performance can
also be observed through Figure 6.12 and Figure 6.13, which depict the complete
progress of the population over time for metrics m5 and m7 respectively. Figure 6.12
shows the fitness of every solution, indexed by generation, with respect to metric m5.
So, for example, in generation 300, almost half the population has maximized m5 to
1, a quarter of the population is very close to 1, and about 10% of the population are
not maximizing m5 at all. Similarly to Figure 6.12, Figure 6.13 represents the fitness
of all generations with respect to m7.
Unlike the previous two scenarios, where once a complete solution is discovered
that solution is further evolved for faster performance, no noticeable performance
increases are guaranteed in this scenario once a complete solution is found, as shown
in the difference between Figure 6.8 and Figure 6.10. Overall, about 50% of the time
a complete solution is evolved that is able to perform quickly. The inability of the
system to consistently evolve a fast solution is attributed to the increased number
of actions and sensor options in this scenario, thus increasing the complexity of the
evolved solution as compared to the previous evolved solutions, and exponentially
increasing the number of candidate solutions.
The faster of the two solutions evolved for object collection is shown in Table 6.12,
and a simplified version of the algorithm is shown in Table 6.13. The simplifications
include the removal of unreachable states and non-firing transitions, and the removal
of actions that will have no effect.
The solution evolved here is much more complex than either of the solutions
evolved for object collection or object destruction. For the most part, the main
86

functionality of the solution is spread across multiple states. Due to the complexity
of the solution and distribution of functionality across many states, it is difficult to
analyze the algorithm completely and extract core behaviors. With this in mind, a
brief analysis of the salient portions of the algorithm will be discussed.
If an agent is in state A and is on a type C object, the agent will pick up the
object and transition to state J. Given a typical run of events, the agent will navigate
to the goal area and deposit the object, cycling through states J → G → K → J.
If an agent is not on a type C object, they will loop through state A until they
are near some type of object. Upon sensing a type C object, the agent transitions to
state J. From here, most likely they will fire the last transition in state J, which leads
to state I, and begin to navigate towards the nearest type C object. From state I the
agent will most likely transition to state C where it will loop until one of the conditions
on the transitions is met. At this point, the serendipity of the agent’s random walk
is relied upon to guide the agent towards the type C object. If the agent does in fact
get to the object, the agent will broadcast the location of the object, transition to
state J and then to state A, where it will pick up the object. From this point, the
agent will deliver the object to the goal area as previously discussed.
In the other case, if the agent starting in state A wanders until it is in the proximity
of a type D object, the agent transitions to state H, broadcasts the location of the
object, and then transitions to state G. From state G, it is difficult to reason about
what transition an agent will follow because there are many possible situations. In
examining simulation runs with only a single agent, the basic result is that the agent
will essentially rely on serendipity to guide the final steps towards the object, then
most likely the object will be acted on in state F reached through state I.
87

6.3 Conclusion
In this chapter, swarm algorithms have been successfully evolved for agent dispersion,
object collection, and object destruction. Composite fitness metrics, created through
the fusion of raw fitness data, and a radix-based ranking algorithm were used to
address the multi-objective nature of the object manipulation problem. In evolving
the object collection and object destruction algorithms, first the core functionality
(collection and destruction, respectively) was evolved. Then, evolution continued
and optimized for execution speed. The object manipulation scenario followed this
trend of first evolving core functionalities, then optimizing for speed. But unlike the
evolution of the object collection and destruction algorithms, the complexity of the
object manipulation problem prevented the consistent evolution of an algorithm that
performed the object manipulation task quickly.
88

Table 6.12: Evolved object manipulation algorithm
A
on-object C pick-up J
near-object C move-to-object D J
near-object D move-random H
holding-object C move-to-object C F
on-goal pick-up G
B
!on-goal move-random J
!on-object D(untouched) move-to-goal K
C
!near-object C broadcast-object D A
on-object D(untouched) broadcast-object D A
holding-object C broadcast-object D D
on-object C broadcast-garbage J
D
near-object D broadcast-object D C
!on-object C move-random K
holding-object C broadcast-garbage D
!holding-object C move-to-object D B
E !near-object D pick-up K
F
on-object C move-to-object C B
holding-object C put-down K
on-object D(damaged) second-attack B
!near-object D ﬁrst-attack B
on-object D(untouched) ﬁrst-attack I
near-object C put-down K
near-object D move-to-object D D
G
on-object D(damaged) broadcast-object D C
holding-object C move-to-goal K
!near-object D move-to-object C B
near-object C broadcast-object D A
!on-goal move-to-goal I
near-object D move-to-object C J
H
!on-object C broadcast-object D G
!on-object D(untouched) move-to-goal H
I
on-goal pick-up E
on-object D(untouched) move-to-object D F
!near-object C broadcast-object D J
on-object C put-down A
!on-object D(damaged) move-to-object C C
near-object C second-attack B
J
on-object D(damaged) pick-up I
holding-object C move-to-goal G
near-object D pick-up I
on-object C broadcast-garbage E
!near-object C put-down C
!on-goal move-to-object C I
K
!on-object D(damaged) put-down J
on-object D(damaged) put-down G
89

Table 6.13: Simpliﬁed version of the evolved object manipulation algorithm
A
on-object C pick-up J
near-object C move-to-object D J
near-object D move-random H
holding-object C move-to-object C F
on-goal - G
B !on-goal move-random J
C
!near-object C broadcast-object D A
on-object D(untouched) broadcast-object D A
holding-object C broadcast-object D D
on-object C broadcast-garbage J
D
near-object D broadcast-object D C
!on-object C move-random K
holding-object C broadcast-garbage D
!holding-object C - B
E !near-object D pick-up K
F
on-object C - B
holding-object C put-down K
on-object D(damaged) second-attack -
!near-object D - B
on-object D(untouched) ﬁrst-attack -
near-object C put-down K
near-object D move-to-object D D
G
on-object D(damaged) broadcast-object D C
holding-object C move-to-goal K
!near-object D move-to-object C B
near-object C broadcast-object D A
!on-goal move-to-goal I
near-object D move-to-object C J
H !on-object C broadcast-object D G
I
on-goal - E
on-object D(untouched) - F
!near-object C broadcast-object D J
on-object C - A
!on-object D(damaged) move-to-object C C
near-object C second-attack -
J
on-object D(damaged) - I
holding-object C move-to-goal G
near-object D pick-up I
on-object C broadcast-garbage E
!near-object C put-down C
!on-goal move-to-object C I
K
!on-object D(damaged) put-down J
on-object D(damaged) - G
90

50 100 150 200 250 300 350
0
0.2
0.4
0.6
0.8
1
Best fitness for Object Manipulation
Generation
NormalizedFitness
m5
m6
m7
m8
t
Figure 6.8: Progress of the best solution over 360 generations for object manipulation is
shown. In generation 325, a solution that maximizes the fitness measures m5–m8 is found.
After this, some progress is made on optimizing the speed of the algorithm, but no real
progress is made.
50 100 150 200 250 300 350 400
0
0.2
0.4
0.6
0.8
1
Mean fitness for Object Manipulation
Generation
NormalizedFitness
m5
m6
m
7
m8
t
Figure 6.9: Mean fitness over time for object manipulation is shown. Notice the effect of
the radix scoring and how neither of the two main tasks (collection or destruction) take a
commanding lead over the other task. Also, unlike the object collection and destruction
evolutions, due to the larger search space, the mean fitness does not begin to decrease after
a good solution is found.
91

50 100 150 200 250 300 350 400
0
0.2
0.4
0.6
0.8
1
Best fitness for Object Manipulation
Generation
NormalizedFitness
m5
m6
m7
m8
t
Figure 6.10: Progress of the best solution over 425 generations for another object manipu-
lation evolution is shown. In generation 160, a solution that maximizes the fitness measures
m5–m8 is found. Around generation 160, a modest amount of progress in time optimization
is achieved, resulting in a solution that is faster than the one evolved in Figure 6.8.
50 100 150 200 250 300 350 400
0
0.2
0.4
0.6
0.8
1
Mean fitness for Object Manipulation
Generation
NormalizedFitness
m5
m6
m
7
m8
t
Figure 6.11: Mean fitness over time for the second object manipulation evolution is shown.
Again, notice the effect of the radix scoring and how neither of the two main tasks (collec-
tion or destruction) take a commanding lead over the other task. Also, unlike the object
collection and destruction evolutions, due to the larger search space, the mean fitness does
not begin to decrease after a good solution is found.
92

0 50 100 150 200 250 300 350 400 450
0
10
20
30
40
50
60
70
80
Distribution of metric m5
scores over each generation
Generation
Solution
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Figure 6.12: The ﬁtness of every solution indexed by generation with respect to metric m5.
0 50 100 150 200 250 300 350 400 450
0
10
20
30
40
50
60
70
80
Distribution of metric m7
scores over each generation
Generation
Solution
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Figure 6.13: The ﬁtness of every solution indexed by generation with respect to metric m7.
93

0 50 100 150 200 250 300 350 400 450
0
10
20
30
40
50
60
70
80
Number of Satisfying Solutions
NormalizedPerformance
Number of solutions satisfying some criteria for Obect Manipulation
m1
m2
m3
m
4
Figure 6.14: Number of solutions satisfying criteria m1, m2, m3, and m4 for object manip-
ulation.
94

Chapter 7
Conclusion
In this work, a general purpose swarm experimentation platform, SWEEP, has been
constructed and interfaced with an evolutionary computing system to demonstrate
the feasibility of autogenerating swarm algorithms given a set of high-level objectives.
The functionality of SWEEP has been validated through the implementation of sev-
eral scenarios including object manipulation and UAV chemical cloud detection. The
utility of fusing SWEEP with an evolutionary computing system has been demon-
strated through the successful evolution of swarm algorithms for dispersion, object
collection, and object destruction. The evolved algorithms exhibited performance
that approximated the runtime performance of hand-coded solutions. The following
sections will summarize what has been demonstrated herein and outline topics for
future work.
7.1 SWEEP
The SWEEP simulation platform developed in this work is an extension of the orig-
inal SWEEP implementation created by Palmer and Hantak [22]. Several vast im-
provements have been made to the core SWEEP functionalities. The parsing of the
simulation speciﬁcation ﬁles has been reimplemented to leverage XML and the object-
95

oriented design of the parsing engine allows for the runtime replacement of any parsing
component. The SWEEP object model has been extended and refined, resulting in a
core set of functionalities that can be extended and customized. Finally, the SWEEP
platform is now better suited to handle larger and more complex problems. Some of
the larger-scale problems that have been addressed using the version of SWEEP de-
veloped in this work, aside from those focused on in this work, include chemical cloud
detection with UAVs [29], using swarm reasoning to address the four-color mapping
problem [47, 40], leveraging group behavior to explore the Martian landscape using
“tumbleweeds” [28], and enhancing swarm algorithm programming through the use
of aspect-oriented programming [30].
7.2 Autogeneration of Swarm Algorithms through
Evolution
This work established the feasibility of autogenerating swarm algorithms through
the implementation of an evolutionary computing algorithm that used a finite state
machine solution encoding and utilized SWEEP as a fitness evaluation tool. Addi-
tionally, this work presented a radix-based ranking mechanism that uses fused fitness
data to address the balancing of multiple evolutionary objectives. Four scenarios were
examined: dispersion, object destruction, object collection, and object manipulation.
For each of the scenarios, the system was able to successfully evolve algorithms that
fully satisfied the fitness criteria. Additionally, the ranking algorithm was successfully
able to balance the evolutionary progress, effectively preventing the survival of expert
solutions that fully address one objective but ignore others.
The use of solely high-level objectives as fitness criteria, though intuitive, does
not make for an effective fitness function. The result is hundreds of generations of in-
effective solutions that produce little or no evolutionary progress. This is because the
96

fitness function does not apply enough pressure to adequately drive selection for both
reproduction and mutation. Selection for reproduction and/or mutation basically
says that some solution has performed well enough, relative to other solutions in that
generation, that its genes should contribute in some capacity to the next generation of
solutions. Implicit in this is that eventually a surviving solution will be replaced by a
better performing solution created through the application of evolutionary operators.
Thus, this implies that small changes made by mutations would have at least a
small but noticeable impact on the fitness of a solution. Through examination of
the state machine representation and the associated mutations, it is clear that small
changes made by a mutation will have little effect on a solution’s fitness when only
accounting for high-level goals. For example, take some generic mutation for a state
machine. If the transition affected by the mutation is never fired, there is no noticeable
change in the fitness of the solution. If the targeted transition is fired, but does not
directly contribute to the achievement of a high-level goal, then again the mutation
is ineffective. Thus, the only way for the mutation to make a noticeable change in
the fitness of a solution is through making a change that directly enables the solution
to improve its ability to achieve a high-level goal. So, starting with a population
of completely random solutions and using only high-level goal information, the only
way for a mutation to affect a positive change in fitness is to apply a random change
to a solution that was randomly generated as almost functional; this event is quite
unlikely.
To address these issues, first the high-level objectives were decomposed into mul-
tiple smaller subgoals. For the individual object collection and object destruction
scenarios, simple radix-based ranking of the subgoals outperformed a weighted rank-
ing scheme in that it prevented the population domination of a sub-optimal solution
with a high-scoring yet low-priority subgoal. Additionally, the splitting of the objec-
tives provided a sufficient granularity in the scoring and ranking to allow the effects
97

of the mutation operators to be more easily observed and exploited by the system.
When addressing the object manipulation scenario, a combination of object col-
lection and destruction, simple radix-based ranking presented a challenge as giving
preference to either the collection or destruction objectives resulted in the evolution of
“expert” solutions that completely addressed one objective but not the other. Thus,
the evolutionary task then became transforming the expert solutions into general
solutions that addressed all the objectives. What was needed was a scoring or rank-
ing method that could balance the evolutionary progress of the population between
multiple independent objectives.
In order to overcome the challenge of balancing the evolutionary progress, a data
fusion technique was employed that used the raw fitness metrics to produce additional
composite metrics, listed in Table 6.6. These composite metrics were compatible with
radix-based ranking and successfully balanced the evolutionary progress between the
collection and destruction fitness metrics, resulting in the system converging to a
complete solution.
7.3 Future Work
The are several paths for future work based on the results presented in this thesis.
Both SWEEP and the evolutionary computing algorithm are plain vanilla implemen-
tations, focusing on clarity and correctness rather than optimal runtime performance.
By extension of the core SWEEP architecture, more computationally efficient meth-
ods of simulation can be implemented. Additionally, a continuation of the work with
aspect-oriented swarm programming [30] can provide SWEEP with an even more
flexible probing mechanism, resulting in the construction of richer fitness functions
for evolution.
As seen through the object manipulation results from Chapter 6, evolution excels
98

at evolving feasible solutions, but can at times struggle with evolving feasible solutions
that are efficient. Further work could investigate the possibility of utilizing evolution
to generate feasible solutions, then piping those solutions into some other optimization
algorithm, such as simulated annealing, for runtime performance optimization. This
would result in the creation of a flexible tool-chain for the generation and optimization
of swarm algorithms.
Currently, the definition of emergent behavior is not standardized, and is primarily
relegated to an “I’ll know it when I see it” method of identification. A formalized
definition of emergent behavior is essential to further work in swarm theory. From this
definition, methods for detecting and measuring the occasion of emergent behaviors
can then begin to be constructed. These methods would be an invaluable addition to
a set of tools for the autogeneration of swarm algorithms. This is perhaps the most
exciting future topic in swarm theory as a complete definition seemingly requires a
truly renaissance approach, spanning from complex systems and information theory
to sociology and philosophy.
99

Appendix A
Example SWEEP Simulation
Speciﬁcation
<?xml version="1.0"?>
<simulation builder="standard.simulation.SimulationBuilder">
<main>
<info name="about" type="info-type1"/>
<swarm name="my-swarm-A" type="swarm-A">
<entity type="swarm-A-entity" number="10"/>
</swarm>
<swarm name="my-swarm-B" type="swarm-B">
<entity type="swarm-B-entity" number="10"/>
</swarm>

<environment name="world" type="world"/>
<probe name="probe" type="terminator"/>
<probe name="summary" type="summarizer"/>
</main>
<environments>
<environment name="world">
<builder class="standard.environment.EnvironmentBuilder"/>
<model class="grid.environment.SafeGridModel">
<attributes>
<attribute name="rows" value="50"/>
<attribute name="cols" value="50"/>
</attributes>
<initializers>
<initializer class="grid.environment.RandomInitialization">
<builder class="grid.environment.RandomInitializationBuilder"/>
<seed value="3141592654"/>

<object value="1" number="50"/>

<object value="4" number="10"/>
</initializer>
</initializers>
<updates/>
</model>
</environment>
</environments>
<models>
100

<model name="avatar" class="grid.model.GridAvatar">
<builder class="standard.model.ModelBuilder"/>

<attributes>

<attribute name="empty" value="0"/>
<attribute name="environment" value="world"/>
<attribute name="holdingGarbage" value="false"/>

<attribute name="garbage.grid" value="world"/>
<attribute name="garbage.value" value="1"/>
<attribute name="garbage.downvalue" value="6"/>
<attribute name="garbage.range.on" value="0"/>
<attribute name="garbage.range.near" value="5"/>

<attribute name="mine.grid" value="world"/>
<attribute name="mine.value.armed" value="4"/>
<attribute name="mine.value.partial" value="3"/>
<attribute name="mine.value.disarmed" value="2"/>
<attribute name="mine.range.on" value="0"/>
<attribute name="mine.range.near" value="5"/>
</attributes>

<sensors>

<sensor name="on-garbage" class="grid.model.sensor.FilteredValueSensor">
<attributes>
<attribute name="environment" value="garbage.grid"/>
<attribute name="range" value="garbage.range.on"/>
<attribute name="targetValue" value="garbage.value"/>
</attributes>
</sensor>

<sensor name="near-garbage" class="grid.model.sensor.FilteredValueSensor">
<attributes>
<attribute name="range" value="garbage.range.near"/>
<attribute name="targetValue" value="garbage.value"/>
<attribute name="varName" value="garbage.location"/>
</attributes>
</sensor>

<sensor name="holding-garbage" class="sweep.model.sensor.InternalBooleanSensor">
<attributes>
<attribute name="attrName" value="holdingGarbage"/>
</attributes>
</sensor>

<sensor name="on-armed-mine" class="grid.model.sensor.FilteredValueSensor">
<attributes>
<attribute name="environment" value="mine.grid"/>
<attribute name="range" value="mine.range.on"/>
<attribute name="targetValue" value="mine.value.armed"/>
</attributes>
</sensor>

<sensor name="on-partially-disarmed-mine" class="grid.model.sensor.FilteredValueSensor">
<attributes>
<attribute name="range" value="mine.range.on"/>
<attribute name="targetValue" value="mine.value.partial"/>
</attributes>
</sensor>

<sensor name="on-dumpster" class="grid.model.sensor.LocationSensor">
<attributes>
<attribute name="location" value="const:10,10"/>
101

</attributes>
</sensor>

<sensor name="near-mine" class="grid.model.sensor.FilteredValueSensor">
<attributes>
<attribute name="range" value="mine.range.near"/>
<attribute name="targetValue" value="mine.value.armed,mine.value.partial"/>
<attribute name="varName" value="mine.location"/>
</attributes>
</sensor>
</sensors>
<actions>

<action name="pick-up-garbage" builder="sweep.model.action.ConditionalActionBuilder">
<if>
<condition sensor="on-garbage" value="true"/>
<condition state="holdingGarbage" value="false"/>
<then>
<action name="pick-up-garbage-helper"/>
<action name="set-holding-garbage"/>
</then>
</if>
</action>

<action name="put-down-garbage" builder="sweep.model.action.ConditionalActionBuilder">
<if>
<condition sensor="on-dumpster" value="true"/>
<condition state="holdingGarbage" value="true"/>
<then>

<action name="reset-holding-garbage"/>
<action name="increment-garbage-count"/>
</then>
</if>
</action>

<action name="pick-up-garbage-helper" class="grid.model.action.SetAction">
<attributes>
<attribute name="value" value="empty"/>
</attributes>
</action>

<action name="put-down-garbage-helper" class="grid.model.action.SetAction">
<attributes>
<attribute name="value" value="garbage.downvalue"/>
</attributes>
</action>

<action name="set-holding-garbage" class="sweep.model.action.SetStateVariableAction">
<attributes>
<attribute name="flag" value="holdingGarbage"/>
<attribute name="value" value="true"/>
</attributes>
</action>

<action name="reset-holding-garbage" class="sweep.model.action.SetStateVariableAction">
<attributes>
<attribute name="flag" value="holdingGarbage"/>
<attribute name="value" value="false"/>
</attributes>
</action>

<action name="increment-garbage-count" class="sweep.model.action.IncrementCounterAction">
<attributes>
<attribute name="name" value="garbage-counter"/>
102

</attributes>
</action>

<action name="disarm-mine-stage-1" builder="sweep.model.action.ConditionalActionBuilder">
<if>
<condition sensor="on-armed-mined" value="true"/>
<then>
<action name="disarm-mine-stage-1-helper"/>
<action name="set-disarmed-phase-1"/>
<action name="disable-agent"/>
</then>
</if>
</action>

<action name="disarm-mine-stage-2" builder="sweep.model.action.ConditionalActionBuilder">
<if>
<condition sensor="on-partial-mine" value="true"/>
<then>
<action name="disarm-mine-stage-2-helper"/>
<action name="set-disarmed-phase-2"/>
<action name="disable-agent"/>
</then>
</if>
</action>

<action name="disarm-mine-stage-1-helper" class="grid.model.action.SetAction">
<attributes>
<attribute name="value" value="mine.value.partial"/>
</attributes>
</action>

<action name="disarm-mine-stage-2-helper" class="grid.model.action.SetAction">
<attributes>
<attribute name="value" value="mine.value.disarmed"/>
</attributes>
</action>

<action name="set-disarmed-phase-1" class="sweep.model.action.IncrementCounterAction">
<attributes>
<attribute name="name" value="disarmed-phase-1"/>
</attributes>
</action>


<action name="set-disarmed-phase-2" class="sweep.model.action.IncrementCounterAction">
<attributes>
<attribute name="name" value="disarmed-phase-2"/>
</attributes>
</action>


<action name="disable-agent" class="sweep.model.action.DisableAction"/>

<action name="move-random" class="grid.model.action.MoveRandomAction">
103

<attributes>
<attribute name="environment" value="environment"/>
</attributes>
</action>

<action name="move-towards-dumpster" class="grid.model.action.MoveTowardsAction">
<attributes>
<attribute name="location" value="const:10,10"/>
</attributes>
</action>

<action name="move-towards-garbage" class="grid.model.action.MoveTowardsAction">
<attributes>
<attribute name="location" value="av:garbage.location"/>
</attributes>
</action>

<action name="move-towards-mine" class="grid.model.action.MoveTowardsAction">
<attributes>
<attribute name="location" value="av:mine.location"/>
</attributes>
</action>
</actions>
<initializers>
<initializer name="random-placement" class="grid.model.RandomPlacementInitializer"/>
</initializers>
</model>
</models>
<controllers>
<controller name="basic-state-machine" format="state">
<builder class="sweep.controller.FSABuilder"/>
<initialState name="searching"/>
<states>

<state name="searching">

<transition nextState="collect-garbage">
<sensor name="near-garbage" value="true"/>
</rule>
<action name="move-towards-garbage"/>
</transition>

<transition nextState="disarm-mine">
<sensor name="near-mine" value="true"/>
</rule>
<action name="move-towards-mine"/>
</transition>

<default nextState="searching">
<action name="move-random"/>
</default>
</state>

<state name="collect-garbage">

<sensor name="holding-garbage" value="false"/>
<sensor name="on-garbage" value="true"/>
</rule>
104

<action name="pick-up-garbage"/>
</transition>

<sensor name="holding-garbage" value="false"/>
<sensor name="near-garbage" value="true"/>
</rule>
<action name="move-towards-garbage"/>
</transition>

<sensor name="holding-garbage" value="true"/>
<sensor name="on-dumpster" value="false"/>
</rule>
<action name="move-towards-dumpster"/>
</transition>

<transition nextState="searching">
<sensor name="holding-garbage" value="true"/>
<sensor name="on-dumpster" value="true"/>
</rule>
<action name="put-down-garbage"/>
</transition>

</default>
</state>

<state name="disarm-mine">

<sensor name="on-armed-mine" value="true"/>
</rule>
<action name="disarm-mine-stage-1"/>
</transition>

<transition nextState="searching">
<sensor name="on-partially-disarmed-mine" value="true"/>
</rule>
<action name="disarm-mine-stage-2"/>
</transition>

<sensor name="near-mine" value="true"/>
</rule>
<action name="move-towards-mine"/>
</transition>

</default>
</state>
</states>
</controller>
</controllers>
105

<agents>
<agent name="agent">
<builder class="standard.agent.AgentBuilder"/>
<attributes/>
<memory/>
</agent>
</agents>
<swarms>
<swarm name="swarm-A">
<builder class="standard.swarm.SwarmBuilder"/>
<entity name="swarm-A-entity">
<agent type="agent"/>
<controller type="basic-state-machine"/>
<model type="avatar"/>
</entity>
</swarm>
<swarm name="swarm-B">
<builder class="standard.swarm.SwarmBuilder"/>
<entity name="swarm-B-entity">
<agent type="agent"/>
<controller type="basic-state-machine"/>
<model type="avatar"/>
</entity>
</swarm>
</swarms>

<information>
<info name="info-type1">
<builder class="sweep.info.NullInfoBuilder"/>
</info>
</information>
<probes>
<probe name="terminator">
<builder class="sweep.probe.CounterProbeBuilder"/>
<iterations num="200"/>
</probe>
<probe name="summarizer">
<builder class="sweep.probe.SummaryProbeBuilder"/>
<sum swarm="my-swarm-A" var="garbage-counter"/>
<sum swarm="my-swarm-A" var="disarmed-phase-1"/>
<sum swarm="my-swarm-A" var="disarmed-phase-2"/>
<sum swarm="my-swarm-B" var="garbage-counter"/>
<sum swarm="my-swarm-B" var="disarmed-phase-1"/>
<sum swarm="my-swarm-B" var="disarmed-phase-2"/>
<screen/>
</probe>
</probes>
</simulation>
106

Appendix B
Evolved Object Destruction
Solution
<?xml version="1.0" encoding="UTF-8"?>
<controller name="controller" format="state">
<initialState name="568fb5:1029f51994d:-7fe8"/>
<states>
<state name="568fb5:1029f51994d:-7fe8">
<transition nextState="568fb5:1029f51994d:-7f57">
<sensor name="on-object-d-untouched" value="true"/>
</rule>
<action name="first-attack"/>
</transition>
<transition nextState="568fb5:1029f51994d:-7f3c">
<sensor name="on-object-d-partial" value="true"/>
</rule>
<action name="second-attack"/>
</transition>
<sensor name="near-object-d" value="false"/>
</rule>
</transition>
</rule>
<action name="broadcast-object-d"/>
</transition>
<transition nextState="568fb5:1029f51994d:-7fe8">
<sensor name="on-object-d-partial" value="false"/>
</rule>
<action name="move-towards-object-d"/>
</transition>
<sensor name="near-object-d" value="true"/>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<transition nextState="568fb5:1029f51994d:-7efc">
</rule>
</transition>
107

<transition nextState="568fb5:1029f51994d:-7fc0">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:1029f51994d:-7fe8">
</default>
</state>
<state name="568fb5:1029f51994d:-7fc0">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<transition nextState="568fb5:1029f51994d:-7e93">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<sensor name="on-object-d-untouched" value="false"/>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<transition nextState="568fb5:1029f51994d:-7e9b">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:1029f51994d:-7fc0">
</default>
</state>
<state name="568fb5:1029f51994d:-7fb1">
</rule>
</transition>
<transition nextState="568fb5:1029f51994d:-7fb1">
</rule>
</transition>
</rule>
</transition>
</rule>
108

</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:1029f51994d:-7fb1">
</default>
</state>
<state name="568fb5:1029f51994d:-7f96">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
109

</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="568fb5:1029f51994d:-7f86">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:1029f51994d:-7fc0">
</default>
</state>
<state name="568fb5:1029f51994d:-7f74">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
110

</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="568fb5:1029f51994d:-7f57">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="568fb5:1029f51994d:-7f3c">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
111

</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="568fb5:1029f51994d:-7efc">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
112

</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:1029f51994d:-7f86">
</default>
</state>
<state name="568fb5:1029f51994d:-7e9b">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
113

</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="568fb5:1029f51994d:-7e93">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
114

</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
</states>
</controller>
115

Appendix C
Evolved Object Collection Solution
<initialState name="568fb5:102d4ad91c4:-7ffe"/>
<states>
<state name="568fb5:102d4ad91c4:-7ffe">
<transition nextState="568fb5:102d4ad91c4:-7f7c">
<sensor name="on-object-c" value="true"/>
</rule>
<action name="pick-up"/>
</transition>
<sensor name="holding-object-c" value="true"/>
</rule>
<action name="put-down"/>
</transition>
<transition nextState="568fb5:102d4ad91c4:-7fc4">
<sensor name="on-object-c" value="false"/>
</rule>
<action name="move-towards-object-c"/>
</transition>
<default nextState="568fb5:102d4ad91c4:-7ffe">
</default>
</state>
<state name="568fb5:102d4ad91c4:-7fde">
<transition nextState="568fb5:102d4ad91c4:-7fd5">
<sensor name="near-object-c" value="false"/>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<transition nextState="568fb5:102d4ad91c4:-7fa5">
<sensor name="on-goal" value="false"/>
</rule>
</transition>
<transition nextState="568fb5:102d4ad91c4:-7f3e">
</rule>
</transition>
</rule>
<action name="move-towards-goal"/>
</transition>
<transition nextState="568fb5:102d4ad91c4:-7f66">
<sensor name="near-object-c" value="true"/>
</rule>
</transition>
<transition nextState="568fb5:102d4ad91c4:-7ede">
<sensor name="holding-object-c" value="false"/>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="568fb5:102d4ad91c4:-7fd5">
<transition nextState="568fb5:102d4ad91c4:-7ebb">
<sensor name="on-goal" value="true"/>
</rule>
</transition>
</rule>
</transition>
116

</rule>
<action name="broadcast-object-c"/>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:102d4ad91c4:-7fde">
</default>
</state>
<state name="568fb5:102d4ad91c4:-7fc4">
<transition nextState="568fb5:102d4ad91c4:-7fde">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="568fb5:102d4ad91c4:-7fa5">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:102d4ad91c4:-7fa5">
</default>
</state>
<state name="568fb5:102d4ad91c4:-7f7c">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:102d4ad91c4:-7fd5">
</default>
</state>
<state name="568fb5:102d4ad91c4:-7f72">
</rule>
</transition>
</rule>
</transition>
117

</rule>
</transition>
<transition nextState="568fb5:102d4ad91c4:-7ffe">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:102d4ad91c4:-7f72">
</default>
</state>
<state name="568fb5:102d4ad91c4:-7f66">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:102d4ad91c4:-7fa5">
</default>
</state>
<state name="568fb5:102d4ad91c4:-7f3e">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
118

</transition>
<default nextState="568fb5:102d4ad91c4:-7f3e">
</default>
</state>
<state name="568fb5:102d4ad91c4:-7ede">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:102d4ad91c4:-7f72">
</default>
</state>
<state name="568fb5:102d4ad91c4:-7ebb">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="568fb5:102d4ad91c4:-7fde">
</default>
</state>
</states>
</controller>
119

Appendix D
Evolved Object Manipulation
Solution
<initialState name="6a3960:102d69ac60b:-7fed"/>
<states>
<state name="6a3960:102d69ac60b:-7fed">
<transition nextState="6a3960:102d69ac60b:-7fd7">
</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7e7d">
</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7fa7">
</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7ed5">
</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7ecd">
</rule>
</transition>
<default nextState="6a3960:102d69ac60b:-7fed">
</default>
</state>
<state name="6a3960:102d69ac60b:-7fd7">
<transition nextState="6a3960:102d69ac60b:-7e53">
</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7f1a">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7e6e">
120

</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7f53">
</rule>
</transition>
</default>
</state>
<state name="6a3960:102d69ac60b:-7fb3">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="6a3960:102d69ac60b:-7fd7">
</default>
</state>
<state name="6a3960:102d69ac60b:-7fa7">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7fed">
</rule>
</transition>
<transition nextState="6a3960:102d69ac60b:-7fb3">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="6a3960:102d69ac60b:-7f53">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
121

</state>
<state name="6a3960:102d69ac60b:-7f1a">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="6a3960:102d69ac60b:-7ed5">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="6a3960:102d69ac60b:-7ecd">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="6a3960:102d69ac60b:-7fa7">
</default>
</state>
<state name="6a3960:102d69ac60b:-7e7d">
</rule>
</transition>
</rule>
122

</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="6a3960:102d69ac60b:-7f1a">
</default>
</state>
<state name="6a3960:102d69ac60b:-7e6e">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</default>
</state>
<state name="6a3960:102d69ac60b:-7e53">
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
</rule>
</transition>
<default nextState="6a3960:102d69ac60b:-7f53">
</default>
</state>
</states>
</controller>
123

Bibliography
[1] Stephen Appleby and Simon Steward. Mobile software agents for control in telecommunication networks. British
Telecom Technical Journal, 18(1):68–70, 2000.
[2] Payman Arabshahi. Swarm intelligence resources page. http://dsp.jpl.nasa.gov/members/payman/swarm, 2002.
[3] Ken Arnold, James Gosling, and David Holmes. The Java(TM) Programming Language (4th Edition). Addison-
Wesley Professional, Reading, MA, 2005.
[4] Thomas Bäck, David Fogel, and Zbigniew Michalewicz, editors. Evolutionary Computation 1: Basic Algorithms
and Operators. Institute of Physics Publishing, Philadelphia, PA, 2000.
[5] Tucker Balch and Ronald Arkin. Behavior-based formation control for multi-robot teams. IEEE Transactions
on Robotics and Automation, 14(6):926–939, 1998.
[6] G. Beni. The concept of cellular robotic systems. In Proceedings 1988 IEEE International Symposium on
Intelligent Control, pages 57–62, Arlington, VA, August 1988. IEEE Computer Society Press.
[7] Eric Bonabeau, Marco Dorigo, and Guy Theraulaz. Swarm Intelligence: From Natural to Artificial Systems.
Oxford University Press, New York, New York, 1999.
[8] Eric Bonabeau and Guy Theraulaz. Swarm smarts. Scientific American, pages 82–90, March 2002.
[9] Timothy Brown, Brian Argrow, Cory Dixon, Sheetalkumar Doshi, Roshan-George Thekkekunnel, and Daniel
Henkel. Ad-hoc UAV ground network (AUGNet). In AIAA 3rd “Unmanned Unlimited” Technical Conference,
Workshop and Exhibit, Chicago, IL, September 2004. American Institute of Aeronautics and Astronautics.
[10] Gianni Di Caro and Marco Dorigo. Antnet: Distributed stigmergetic control for communications networks.
Journal of Artificial Intelligence Research, 9:317–365, 1998.
[11] Bruce Clough. Personal communication. Wright-Patterson AFB Air Force Research Lab, 2002.
[12] Bruce Clough. UAV swarming? So what are those swarms, what are the implications, and how do we handle
them? Technical Report AFRL-VA-WP-2002-308, Wright-Patterson AFB Air Force Research Lab, 2002.
[13] Thomas Cormen, Charles Leiserson, and Ronald Rivest. Introduction to Algorithms. The MIT Press, Cambridge,
MA, 1989.
[14] Iain D. Couzin, Jens Krause, Richard James, Graeme D. Ruxton, and Nigel R. Franks. Collective memory and
spatial sorting in animal groups. Journal of Theoretical Biology, 218:1–11, 2002.
[15] J. Alexander Fax and Richard M. Murray. Information flow and cooperative control of vehicle formations. IEEE
Transactions on Automatic Control, 49(9):1465–1476, September 2004.
[16] Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design Patterns: Elements of Reusable
Object-Oriented Software. Addison-Wesley, Reading, MA, 1994.
[17] Paul Gaudiano, Eric Bonabeau, and Ben Shargel. Evolving behaviors for a swarm of unmanned air vehicles. In
IEEE Swarm Intelligence Symposium, Pasadena, California, June 2005.
[18] Veysel Gazi and Kevin Passino. Stability analysis of swarms. IEEE Trans. on Automatic Control, 48(4):692–697,
April 2003.
124

[19] Jared Go, Thuc Vu, and James J. Kuffner. Autonomous behaviors for interactive vehicle animations. In SCA
’04: Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 9–18,
New York, NY, USA, 2004. ACM Press.
[20] P.-P. Grassè. La reconstruction du nid et les coordinations inter-individuelles chez Bellicositermes natalensis et
Cubitermes sp. La thiorie de la Stigmergie: Essai d’interpritation du comportement des Termites Constructeurs.
Insectes sociaux, 6:41–80, 1959.
[21] D. Grunbaum, S. Viscido, and J. K. Parrish. Extracting interactive control algorithms from group dynamics of
schooling fish. In V. Kumar, N. E. Leonard, and A. S. Morse, editors, Lecture Notes in Control and Information
Sciences, pages 103–117. Springer-Verlag, 2004.
[22] Chad Hantak and Daniel Palmer. Implementing a swarm intelligence software laboratory using finite state
automata. In Proceedings 12th National Conference on Undergraduate Research (NCUR), volume 3, pages
984–988, Salisbury, MD, 1998. National Conference on Undergraduate Research.
[23] John Holland. Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor,
1975.
[24] Cristian Huepe and Maximino Aldana. Intermittency and clustering in a system of self-driven particles. Physical
Review Letters, 92(16):168701, April 2004.
[25] Major General Kenneth R. Israel. Modeling and simulation employed in the Predator unmanned aerial vehicle
program. http://www.fas.org/irp/agency/daro/product/predms.htm, March 1997.
[26] Ali Jadbabaie, Jie Lin, and A. Stephen Morse. Coordination of groups of mobile autonomous agents using nearest
neighbor rules. IEEE Transactions on Automatic Control, 48(6):988–1001, June 2003.
[27] Dan Koeppel. Massive attack. Popular Science, pages 38–44, December 2002.
[28] Richard. M. Kolacinski, Daniel W. Palmer, Patrick M. Cloutier, and Jason E. Schatz. Biologically inspired
design for low cost exploration of space: Swarms of martian rovers based on the russian thistle. In Proceedings
of the 7th World Multiconference on Systemics, Cybernetics and Informatics, volume 11, pages 32–38, Orlando,
FL, July 2003.
[29] Michael A. Kovacina, Daniel W. Palmer, Ravi Vaidyanathan, and Richard M. Kolacinski. Intelligent/adaptive
operators and representation modulation for evolutionary programming. In AIAA 3rd Unmanned Unlimited
Technical Conference, Chicago, IL, September 2004. American Institute of Aeronautics and Astronautics.
[30] Peter T. Kovacina. Use of aspect-oriented programming with swarm algorithms. In Proceedings 19th National
Conference on Undergraduate Research (NCUR), Lexington, VA, April 2005. National Conference on Under-
graduate Research.
[31] John R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT
Press, Cambridge, MA, USA, 1992.
[32] M. Anthony Lewis and George Bekey. The behavioral self-organization of nanorobots using local rules. In
Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, volume 2, pages 1333–
1338, July 1992.
[33] Peter Linz. An Introduction to Formal Languages and Automata. D. C. Heath and Company, Lexington, MA,
USA, 1990.
[34] Joshua A. Marshall, Mireille E. Broucke, and Bruce A. Francis. Formations of vehicles in cyclic pursuit. IEEE
Transactions on Automatic Control, 49(11):1963–1974, November 2004.
[35] Orbital Research Inc. Auto-adaptive fuzzy rule base control of autonomous swarms. SBIR Phase I Final Report,
Kirtland AFB Space Vehicles Directorate, 1999.
[36] D. Paley, N. E. Leonard, and R. Sepulchre. Collective motion: Bistability and trajectory tracking. In IEEE
Conference on Decision and Control, 2004.
[37] Daniel Palmer, Chad Hantak, and Michael Kovacina. Impact of behavior influence on decentralized control
strategies for swarms of simple, autonomous mobile agents. In Workshop: Biomechanics Meets Robotics Modeling
and Simulation of Motion, Heidelberg, Germany, November 1999.
125

[38] Daniel Palmer, Marc Kirschenbaum, Jon Muron, and Kelly Zajac. Decentralized cooperative auction for multiple
agent task allocation using synchronized random number generators. In Proceedings of the 2003 International
Conference on Intelligent Robots and Systems, pages 1963–1969, Las Vegas, NV, October 2003.
[39] Daniel Palmer, Marc Kirschenbaum, Jon Murton, Michael Kovacina, Daniel Steinberg, Sam Calabrese, Kelly
Zajac, Chad Hantak, and Jason Schatz. Using a collection of humans as an execution testbed for swarm
algorithms. In Proceedings of the IEEE Swarm Intelligence Symposium, pages 58–64, Indianapolis, IN, April
2003. Institute of Electrical and Electronics Engineers.
[40] Daniel Palmer, Marc Kirschenbaum, Jason Shifflet, and Linda Seiter. Swarm reasoning. In IEEE Swarm
Intelligence Symposium, Pasadena, CA, June 2005.
[41] Julia K. Parrish, Steven V. Viscido, and Daniel Grunbaum. Self-organized fish schools: An examination of
emergent properties. Biol. Bull., 202:296–305, June 2002.
[42] H.V.D. Parunak, M. Purcell, and R. Connell. Digital pheromones for autonomous coordination of swarming
uavs. In Proceedings of the AIAA’s First Technical Conference and Workshop on Unmanned Aerospace Vehicles,
Systems, and Operations, 2002.
[43] Jan Willem Polderman and Jan C. Willems. Introduction to Mathematical Systems Theory: A Behavioral
Approach. Springer-Verlag, Secaucus, NJ, USA, 1998.
[44] Craig Reynolds. Boids (flocks, herds, and schools: A distributed behavior model). http://www.red3d.com/cwr/
boids, June 2001.
[45] Craig W. Reynolds. Flocks, herds, and schools: A distributed behavioral model. Computer Graphics, 21(4):25–
34, 1987.
[46] Bruce Schechter. Birds of a feather. New Scientist, pages 30–33, January 1999.
[47] Jason Shifflet. Swarm-based reasoning for the four color mapping problem. In Proceedings 19th National Confer-
ence on Undergraduate Research (NCUR), Lexington, VA, April 2005. National Conference on Undergraduate
Research.
[48] Robert Tobias and Carole Hofmann. Evaluation of free Java-libraries for social-scientific agent based simulation.
Journal of Artificial Societies and Social Simulation, 7(1), January 2004.
[49] John Toner and Yuhai Tu. Flocks, herds, and schools: A quantitative theory of flocking. Physical Review E,
58(4):4828–4858, October 1998.
[50] Chad M. Topaz and Andrea L. Bertozzi. Swarming patterns in a two-dimensional kinematic model for biological
groups. SIAM Journal of Applied Math, 65(1):152–74, 2004.
[51] M. Trahan, J. Wagner, K. Stantz, P. Gray, and R. Robinett. Swarms of UAVs and fighter aircraft. In Proceedings
of the Second International Conference on Nonlinear Problems in Aviation and Aerospace, volume 2, pages 745–
752, Daytona Beach, FL, April 1998.
[52] Tamas Vicsek, Andras Czirok, Eshel Ben-Jacob, Inon Cohen, and Ofer Shochet. Novel type of phase transition
in a system of self-driven particles. Physical Review Letters, 75(6):1226–1229, August 1995.
[53] Gang Yang and Vikram Kapila. Optimal path planning for unmanned air vehicles with kinematic and tactical
constraints. pages 1301–1306, Las Vegas, NV, December 2002. Institute of Electrical and Electronics Engineers,
Proceedings of the IEEE Conference on Decision and Control.
[54] Nahum Zaera, Dave Cliff, and Janet Bruten. (Not) evolving collective behaviours in synthetic fish. Technical
Report HPL-96-04, Hewlett-Packard Laboratories, 1996.
126

Mak ms

More Related Content

What's hot (15)

Viewers also liked (9)

Similar to Mak ms (20)

Mak ms