A Research Platform for Coevolving Agents.doc
- 1. A Research Platform for Coevolving Agents with Producer/Consumer Interactions R. E. Smith The Intelligent Computing Systems Centre The University of The West of England robert.smith@uwe.ac.uk Introduction This report documents progress made during the author's summer research fellowship in the Intelligent Business Systems Group at BT Research Laboratories. The focus of the fellowship was on the exploration of evolutionary computation (EC) in agent-based systems, with a mind towards applications of interest to BT research. The work built on a framework for EC in mobile agents that the author had constructed in anticipation of the fellowship (Smith & Taylor, 1998). The result of the fellowship is a platform for examining coevolving agents in a controllable environment. This environment also has a direct relationship to other agent-based system efforts going on at BT, under the supervision of Dr. Paul Kearney. The platform is an extensible system where agents interact in a producer/consumer economic world. This report describes the motivations for this system, its design, preliminary results, and directions for future research. Motivations for the EC Agents Framework Agent-based systems are of particular interest in a variety of telecommunications applications. The key qualities that agent-based components and systems exhibit are: autonomy, reactivity, proactivity, and social behavior. Moreover, agents have the possibility of mobility in complex network environments, putting software functions near the computational resources they require. Agents can also explicitly exploit the availability of distributed, parallel computation facilities (Franklin & Graesser, 1997; Wooldridge & Jennings, 1996). However, these qualities ultimately depend on the potential for agent adaptation. For instance, if an agent is to operate with true autonomy in a complex environment, it may have to react to a spectrum of circumstances that cannot be foreseen by the agent’s designer. Autonomous agents may need to explore alternative reactive and proactive strategies, evaluate their performance online, and formulate new, innovative strategies without user intervention. Moreover, for systems of agents to behave in this manner, social interactions between agents may also need to adapt and emerge as conditions change. Areas where agents could benefit from adaptation are addressed by active research in machine learning (e.g., classification of unforeseen inputs, strategy acquisition through reinforcement learning, etc.). However, many machine learning techniques are focused on centralized processing of databases to formulate models or strategies. In contrast, evolutionary computation (EC) techniques are inherently based on a distributed paradigm (natural evolution), making them particularly well suited for adaptation in agents. The
- 2. following sections provide further motivation for investigating EC in agent-based systems. Empirical Motivation Consider the author's previous work on acquiring novel combat maneuver strategies for fighter planes (Smith & Dike, 1995) using genetics-based machine learning. In this project, systems of distinct rules coevolve under the action of an EC system to specify complex maneuvers, like the one shown in Figure 1. In this maneuver, the plane on the right is controlled by the genetics-based machine learning system, while the plane on the left is following standard combat logic. Both planes execute a change in their controls every 1/10th of a second. Each control action is specified by a separate rule. Note that rules firing early in the maneuver sacrifice immediate advantage (turning nose away from the opponent) such that later rules can dictate a tight turn, and overwhelm the opponent. This demonstrates that although the genetically-learned rules evolve as separate entities (agents), they exhibit complex cooperative behaviors. Figure 1: A maneuver dictated by a set of rules, coevolved in an evolutionary computation system (Smith & Dike, 1995). This result shows that complex, innovative, multi-component adaptive systems can emerge from EC processes. Theoretical Motivation Although the empirical evidence is compelling, one must ask whether there is a more mathematically justifiable reason for examining EC in agent-based systems. To provide and answer, consider the oft-cited motivations for agent-based systems themselves: • proactivity • reactivity • social interaction
- 3. • autonomy • distributability Each of these motivations, to some extent, suggests adaptation or learning. However, when one considers agent-based systems, and social interaction, one must consider the impact of learning on the system of agents, while maintaining distributability. In other words, one must consider how to obtain and propagate learned information in a system of agents, without a centralized coordinator of this activity. Holland’s seminal book, Adaptation in Natural and Artificial Systems (Holland, 1975) was primarily addressed at this issue. Although the book is often cited as the first on genetic algorithms (GAs), one of the earliest forms of EC, it is much more directed at general issues of adaptation in systems of distributed entities. The book mathematically examines how populations of entities interacting in “systems” can and should adapt. Extending this to agents is a natural implication of Holland's work. There are four main theoretical pillars upon which Holland's theories are based: • The K-armed Bandit • The Schema Theorem • The Building Block Hypothesis • Implicit Parallelism To understand these motivations, consider an agent interacting with other agents and entities in a complex environment. Assume each agent is made up of some sort of encoded feature set. Given this assumption, one can ask the general question of how one should determine features to try in other agents? Clearly, there is seldom a clear mapping from these features to agent fitness. However, Holland manages to deal with this question in a general context by casting it as a probabilistic decision problem: the k-armed bandit. Consider two alternative “features” that an agent may have. Given a feature’s interactions with other features within the agent, with other agents, and with a complex environment one has, at best, a noisy evaluation of each alternative feature. Therefore, to evaluate the feature, one can spend some time evaluating the alternative features in a variety of agents and contexts. However, this leads to a potential loss in agent performance while trying out features that turn out to be less advantageous. So, it is desirable to quickly incorporate features that appear advantageous into our agents. However, this leads to a potential performance loss if a less advantageous feature is selected prematurely. This is the classic dilemma of exploration versus exploitation. One can overcome this dilemma by increasing the number of agents with the apparent best feature gradually while continuing to experiment with other candidates. However, one has a choice with regard to the speed of this feature propagation. As a function of time (t), should one increase the number of agents (m(H,t)) with the apparent best feature (H): • linearly: m( H , t + 1) = m( H , t ) + C geometrically: m( H , t + 1) = m( H , t ) + Ct D •
- 4. • exponentially: m( H , t + 1) = Cm( H , t ) , or superexponentially: m( H , t + 1) = Ct m( H , t ) D • where C and D are constants? Holland’s K-armed bandit argument shows that, regardless of the distribution of noise surrounding the feature’s fitness, one should allocate the apparent best feature exponentially, to insure minimum loss. Moreover, Holland's Schema Theorem of GAs shows that a reproductive (EC) plan does just such an allocation. This leads to Holland's Building Block Hypothesis, which is the central heuristic in GA search. This heuristic assumes that if one can propagate advantageous features (building blocks) in a population of agents, they will combine (build) into improved agents, and agent system behaviors. Although this heuristic is clearly not adequate for all situations, it seems appropriate, given a general system with a distributed character. Moreover, Holland shows that in considering a population of agents, one implicitly applies the k-armed bandit argument, through the schema theorem, to a large number of building blocks to which the building block hypothesis might accurately apply. Clearly, one cannot expose each and every feature to the same exponential decision process, since the space of features (where a feature is any combination of elements with an agent) is far larger than the space of possible agents. . However, Holland’s implicit parallelism argument shows that for moderate population sizes N, the number being processed is on the order of N3. Given these empirical and theoretical arguments, Holland’s genetic adaptive plans seem the appropriate way to propagate features through a system of agents. Moreover, these EC processes implicitly exploit parallelism, while remaining trivial to explicitly parallelize. Therefore, EC methods are one of the most natural machine learning techniques to transfer general-purpose adaptive capabilities to agent-based systems. Philosophical Motivation The previous sections outline motivations for exploring EC in agent-based systems. In many ways, all research on coevolving EC systems is apropos to agent-based systems. However, EC systems that use actual software agents have been rare. One possible exception is work on Tierra (Ray, 1990), which involves the coevolution of pieces of computer code. However, Tierra "agents" are not standard software components. They cannot perform the actions of general software agents, and require an EC specific software environment in which to run. It is important to investigate EC (and other distributed AI techniques) within an environment of real-world, standards-based software components, for reasons of embodiment (Brooks, et al., 1998). In the area of robotic artificial intelligence, embodiment means that a robot's sensory-motor systems cannot be fully disconnected from behavior, capability to adapt, and intelligence. Internal representations, reasoning, and behavior are ultimately grounded in the physical nature of the robot, and its physical world. Therefore, Brooks and others are addressing the issue of robotic intelligence through hardware robots, where embodiment is considered directly.
- 5. Similar arguments can be made for artificial intelligence in agent-based systems. To consider the embodiment of adaptation, behavior, and intelligence in such agents, one must examine real sensors, effectors, and environments along with those agents. Thus, it is important to consider EC in systems of standards-based software agents, that interact in a generalized computing environment, and are not specifically designed for EC experiments.. Pre-BT Fellowship Results Until recent work by the author, no standards-based, EC agent framework existed. Before the BT fellowship, the author had developed an agent-based EC framework. There are several potential advantages to this framework. The primary is advantage is that such a framework could allow several users or agents, at distributed locations, to interact in EC processes in a standardized way, without a priori knowledge of these processes. Potential applications include collaborative search, and collaborative problem solving. Details of the framework itself are included in this report's Appendix (Page 16). The framework was tested on problems where coevolution was not necessary (i.e., optimization problems). Results from one such experiment are shown in Figure 2. 30 25 Agent Fitness 20 15 10 5 0 0.0E+00 1.0E+06 2.0E+05 4.0E+05 6.0E+05 8.0E+05 1.2E+06 Tim e (m illiseconds) Figure 2: A result with the EC agent framework applied to a simple optimization task. In this experiment, each agent was set with the simple task of finding fit mates. Every agent advertises (through a Plumage object) its own fitness as the number of ones in an internal, 32-bit chromosome. Every agent trusts the fitness advertised by other agents, and as a result the number of agents with the "all ones" chromosome gradually increases. Note that such trust is not necessary for the general operation of the framework. In other words, this experiment is an agent-based implementation of an EC benchmarking scenario known as the 32-bit "counting ones" problem. All agents play both a paternal and maternal role. Once an agent "mothers" a child, it dies, thus insuring a constant
- 6. population size. One also can think of each matrilineal trace as being a single agent that can modify its own characteristics. The result in Figure 2 is similar to those of standard EC systems, despite the asynchronous, decentralized, agent-based nature of the system. The distributed nature of the experiment is better illustrated in Figure 3 30 25 Agent Fitness 20 15 10 5 0 160 78050 123250 184000 237990 315540 376180 441490 523820 618400 689860 803830 860240 938730 987340 Agent Generated (Tim e Labels) (m illiseconds) Figure 3: A single maternal line, from the experiment shown in Figure 2. The figure shows the fitness of a single "maternal line". Imagine that this line of agents remains the "property" of a single user. This user could have genetic interactions with other users, through the framework. Such interactions would improve those agents belonging to the user. Although these results are encouraging, they do not consider a key issue in both EC and agent-based systems research. The fitness value of each agent in the previous experiment is independent of the fitness values of the other agents in the population. Moreover, their only interactions are genetic. The goal of the BT fellowship research was to begin a deeper investigation, where evolving agents were interdependent. Such efforts should relate to previous centralized genetics-based machine learning systems (like those in Smith & Dike, 1995), and more realistic agent based systems. Moreover this research will being to address the watchwords of modern EC research, including niches, coevolution, and emergence. There are proven theories on how an EC system can implicitly balance a diverse, cooperative population, while exploiting Holland’s theories (Deb, 89; Smith, Forrest, & Perelson, 95; Horn & Goldberg, 96). However, these issues have yet to be evaluated in a standard framework of asynchronous software agents. The BT work has set the stage for this examination, in a producer/consumer agent world.
- 7. The Producer/Consumer Interaction Framework Much work has been done in the area of coevolution in EC systems (Rosin & Belew, 1997). However, many of the systems in past research are difficult to parameterize for detailed examination of various effects. In consultation during the BT fellowship, Paul Kearney, Walter Merlat, and the author developed a much more tractable system for these examinations, based on producer/consumer economics. The problem setting is as follows. Any number of agents interact in a specific economy. Each agent in the system has a pre-specified number of workers at its disposal. These workers can be allocated to any one of M possible technologies. There are N possible goods in the economy. Each technology converts one set of goods into another set. Technologies are a pre-specified aspect of the economic world. From an agent's perspective, the goal is to allocate its workers, and maximize its profit. An agent must perform this task, given that the price of each good varies with time, with the action of other agents, and with external market forces. Each of these price variations takes place through a market process of supply and demand. From the perspective of the overall system, one goal is to demonstrate the evolutionary emergence of interesting, productive economic interaction amongst selfish, co-evolved agents. Another goal is to examine variations in emergent behavior, based on variations in system parameters, and individual agent behavior. Software Structure The system designed during the BT fellowship is intended as a platform for a wide variety of ongoing experiments. Therefore, it was important to maintain an extensible, object-oriented design. The following are the key classes in that design. Javadoc and source code files are available within the BT Intelligent Business Systems Research Group. Like the EC Framework, the agent classes here are built around IBM's Aglets. However, they could be easily transferred to other agent systems. Each class is extensible for future experimentation. The marketAgent class Agents of this class process orders for goods from producer/consumer agents, and credit these agents’ cash account. The economicWorld class This class defines the "rules" of the economic world, including • M, the number of technologies • N, the number of goods • the set of technologies, and what they do, and • external consumers and producers (of end goods and raw materials, respectively). The producerAgent class This is the key agent class. Each producerAgent responds to “Current Prices” messages from agent(s) of the marketAgent class. The producerAgent determines a
- 8. worker allocation based on this current information, and its own Genotype object (see details of the EC agent framework in the Appendix, Page 16). The producerAgent then submits an order message to the marketAgent, with negative and positive quantities representing sell and buy orders for each good. When triggered by an internally defined condition, a producerAgent searches out mates, and attempts to produce child producerAgents. Each of these basic classes is extensible for a number of market and genetic behaviors. Also, there are a set of utility classes that allow for graphical interaction with an experiment. Screen shots are shown in Figure 4. Figure 4: Screen shots from the Java-based producer/consumer system. Preliminary Experiments To illustrate the basic operation of the producer/consumer economic framework, a simplified economic world was constructed. The Market In this world, there is only one marketAgent, which maintains a store of each good. Each store can have positive and negative quantities, and represents the excess supply of a good. The effect of supply and demand is simulated, by basing each good's price, P, on a simple function of the amount in the marketAgent's excess store of that good, S: P = exp( − λ S )
- 9. where λ is a parameter. Prices are only recalculated at the end of a trading day. A trading day begins with the marketAgent broadcasting a current price message to every producerAgent. The market then asynchronously processes orders from each producerAgent that received this current price message. The trading day is complete when all these orders are processed. Prices are then recalculated, and a new trading day begins. The Agents In this preliminary experiment, the producerAgents are also simplified. In its Genotype, each producerAgent has a BooleanGene for each of the M technologies. A producerAgent allocates its workers evenly to each technology that is marked true in its Genotype. In the preliminary experiment, producerAgents trade for 10 trading days, then they (asynchronously) broadcast and receive Plumage and Sperm objects, to locate mates. In this case, a Plumage object contains the broadcasting agent's evaluation of its own profit per trading day, which is implicitly trusted by the other agents. Note that such trust is an aspect of this experiment only, and is not, in general, a necessary aspect of the framework. When five or more Plumage objects are received by an agent, the best is picked, and the associated Sperm object is used to create a producerAgent child. Then the maternal parent agent dies. Note that in the paternal role, an agent can have many children, based on it's relative profit potential, while in the maternal role, an agent only has one child. The Economic World In the preliminary experiment, it was desirable to have a set of technologies and goods that interacted in a uniform, extensible manner. Therefore, the following scheme was employed. Goods were "stacked" into R rows and C columns, as shown in Figure 5.
- 10. End goods out good(R,0) good(R,1) good(R,2) ... good(R,C) ... ... ... ... good(1,0) good(1,1) good(1,2) ... good(1,C) good(0,0) good(0,1) good(0,2) ... good(0,C) good(0,C) Raw materials in Figure 5: The simplified economic world used in preliminary experiments. In this figure, the unfilled arrows represent technologies. Note that each technology takes two goods in one row, and converts them into a single good in the next row. Such conversions are conservative, such that x quantity of one good, combined with y quantity of another good, yields (x+y) quantity of the third good. Note that the world "wraps around" at the right and left edges. Thus, the world is expandable via the R and C parameters, and goods in the R-2 intermediate rows are treated identically. The filled arrows in Figure 5 represent external suppliers of raw materials, and consumers of end goods. The raw material goods (in the lowest row) and the end goods (in the highest row) are controllable through the supply and demand of external suppliers and consumers. Note that the total number of goods N=R*C, and the total number of technologies M=(R-1)*C. The preliminary experiment presented here is the minimal meaningful form of this world. There is one layer of intermediate goods (R=3). For simplicity, C=3, giving a total of N=9 goods and M=6 technologies. External supplies and demands are manipulated such that there is a fixed price for the raw materials, and another fixed price for end goods. Results In the first experiment, the prices of these raw materials and end goods are both maintained at a value of one (i.e. the external supply and demand adjust so as to maintain S = 0 for these goods). A population of 50 producerAgents interacts in the market, and in the EC mating process. Prices of the three intermediate goods in this simulation are shown in Figure 6. Variations in these prices are due to evolution of the agents involved in the market. As prices for intermediate goods rise (due to low supply and high demand), agents evolve to produce these high-profit goods. This causes supply to increase, demand to fall, and prices to fall.
- 11. Then the agents evolve towards being consumers of the low-cost intermediate goods. Note that the agents involved have no memory of price history. All changes in behavior occur through the action of EC. 18 16 14 12 10 Prices 8 6 4 2 0 1 169 337 505 673 841 1177 1345 1513 1681 1849 2017 2185 2353 2521 2689 2857 1009 Trading Day Figure 6: Prices of three intermediate goods as a function of trading day in an evolving economy of producers and consumers. In a second experiment, the price of raw materials remains fixed at 1, but the price of end goods is raised to the fixed value of 5. Thus, a fixed price gradient is maintained from raw materials to end goods. Prices of the three intermediate goods in this simulation are shown in Figure 7. Once again, all price variations are due to agent evolution. 50 45 40 35 30 Price 25 20 15 10 5 0 364 727 1 122 243 485 606 848 969 1090 1211 1332 1453 1574 1695 1816 Trading Day Figure 7: Prices of three intermediate goods as a function of trading day in an evolving economy of producers and consumers, with a fixed, positive price gradient.
- 12. Analysis Clearly, these are only preliminary results from the system. However, they do point to the advantages of the framework developed during the BT fellowship. Unlike much of the extant research on coevolving systems (for instance, the fighter plane research shown in Figure 1), this particular setting yields results can be analyzed. For instance, consider a Fast Fourier Transform of the data in Figure 6, shown as a power spectral density in Figure 8. In this figure, the power spectrum for each good is normalized, such that the maximum power equals one. 1 Relative 0.5 Power 0 1024 512 341 256 205 171 146 128 114 102 93 1/Frequency (wavelength in trading days) Figure 8: A power spectral density of prices for the three intermediate goods shown in Figure 6. Power values are normalized, such that the maximum power for each good equals one. The graph shows the complexity of the price fluctuations for each good (i.e., there are many significant frequency components) Note that, given the symmetries of the economic world, there is no clear reason that each good should have different price variations. However, although the goods all share some frequencies of variation in common (chiefly those around 1/300 trading days), there are broad variations in the secondary frequency components. This is most likely due to subtle, "genetic drift" effects in the EC system. Contrast these results to those shown in Figure 9, which is a similar power spectral density for the experiment shown in Figure 7. Recall that in this experiment, there is a positive price gradient maintained from the raw materials to the end goods. The power spectral density shows that the price variations in this experiment are much less complex than those in the previous experiment (i.e., there are fewer pronounced secondary frequencies). Moreover, the price variations in the intermediate goods (although different in scale) are more similar than those of the previous experiment (i.e., there are fewer differences between the spectra of the different goods).
- 13. 1 Relative 0.5 Power 0 1024 512 341 256 205 171 146 128 114 102 93 1/Frequency (wavelength in trading days) Figure 9: A power spectral density of prices for the three intermediate goods shown in Figure 7. Power values are normalized, such that the maximum power for each good equals one. Future Study Clearly, these experiments only begin to scratch the surface of a large area for investigation that is of importance to both the EC and the agent-based systems communities. The research environment presented here is uniquely suited to this investigation. It has the advantages of • embodiment of the agents in a generalized, fully asynchronous agent system (IBM's Aglets), • a flexible, object-oriented design, well-suited to modifications, extensions, and further experimentation, • possible extension into real-world applications, and • coevolved results that manifest themselves in a clear, analyzable form. The last point is key: previous research efforts on co-evolved systems have used experimental environments where clear-cut examination of agent interactions was difficult or impossible. Several further experiments with the current system present themselves immediately, including those discussed below. Problem Variations In the preliminary experiments shown here, supply and demand are manipulated to maintain constant prices for raw materials and end goods. However, a more realistic simulation would consider fixed flow rates for raw materials and end goods, or flows that
- 14. are dependent on prices. This is a fertile area for exploration, that is easily supported by the current framework. Another problem variation lies in the complexity of technology interactions. In the preliminary experiments presented here, there is a uniformity of the economic world, induced by the similarity of all technologies. Breaking symmetries in the economic world, through the introduction of more complex technologies, is an interesting area for investigation. Once again, this extension is easily introduced into the current framework. Agent Variations The most significant simplification in the preliminary experiments is that the agents assign their workers blindly, as dictated by their genetic code. All reactivity to price patterns occurs on the evolutionary time scale. An obvious next step is to build simple price reactivity to the agents. As a first step, this would involve no learning of price patterns during the agent's lifetime. The agent would only have the capacity to shift worker allocation in reaction to current prices, due to a strategy dictated by its genetic code. This closely parallel's previous work by the author on coevolution in learning classifier systems (Smith, 1995). A second step would by to allow the agents to adapt internal parameters during their lifetimes. This would involve memory of past price patterns within the agent, and resulting lifetime learning. Through such learning, an individual agent would have the capacity to predict and anticipate price patterns. Once again, this closely parallels previous work by the author (Smith, 1995). Note that as the complexity of the agent's strategies increases, issues of embodiment become more important. An agent’s ability to sense, calculate, and execute are implicitly tied to its computational interactions with other agents, and interactions with its computational environment. In these situations, the sort of framework presented here is key to future investigation. The current framework will easily support these extensions. EC Variations The modifications suggested above lead to important issues in EC. Chief among these is the Baldwin Effect. This effect is the interplay of lifetime learning and genetic evolution. If an agent is evolved to learn a particular behavioral strategy, that agent's descendents are more likely to learn that strategy as well, although the strategy is not specifically inherited genetically. This interplay of genetic learning and lifetime learning is key to desirable emergent behavior in agent-based systems. The current system is well poised for an examination in this area. Many other more technical EC issues can be examined within the current system, including: • more complex coding of strategies, including genetic programming techniques • mating restrictions to promote distinct species of agents • diploidy and dominance to improve tracking of time variations in the agent-based system (Smith, 92)
- 15. Given the object based nature of the EC agents framework, each of these extensions is straight forward. Final Comments The system of producer/consumer agents developed during the BT fellowship, coupled with the author's previous work on constructing a framework for EC in agent-based systems, presents a clear opportunity for further research. The current system is analyzable, while maintaining a realistic, embodied character of the agents. Preliminary results show the promise of investigations using this system. Given that EC seems a theoretically and empirically appropriate technique for developing social, adaptive behavior in agents, further investigations with this system are promising.
- 16. Appendix: Details of the EC Agents Framework Design Philosophy The structure of a typical EC software system is shown in Figure 10. In such systems, a central program manipulates data structures that comprise the GA population. Although these structures may act as agents, through interactions with an external world, they do not function autonomously. All interactions are dictated by the central GA program. Even in distributed GAs (Kapsalis, Smith & Rayward-Smith, 1994) this structure is maintained, though the central GA program may be replicated at several locations. Centralized GA program Evaluation/Interaction population w/ external applications Selection Recombination Mutation Individuals are simple genetic encodings of parameters Figure 10: The structure of typical GA (EC) software systems. A truly agent-based GA structure is shown in Figure 11. In this system, the population is comprised of standardized software agents, that interact with each other, and with the external world, autonomously. There is no central GA. Agents can interact in general ways, not just genetically. When interacting genetically, these agents exchange standard objects that allow them to select mates, exchange genetic material, and construct children, all based on their own internal goals and procedures.
- 17. Generalized Agent Environment contains genetic and non-genetic agents Agent’s Evaluation and Selection of Mates Agent’s genetic and non-genetic Recombination Strategy message interactions with other agents Agent’s and the “outside world” Mutation Strategy “Child” Agent Figure 11 Implementation Specifics The EC agents framework was implemented in Java, for its machine independence. The framework is also based on IBM's Aglets, for agent infrastructure, relationship to emerging systems standards, and the possibility of agent mobility. Javadoc is available for the entire framework, from the Intelligent Business Systems Group at BT Labs. The following sections outline the basics of the framework. The EC agents framework consists of two main packages. The ECbasics package defines a set of extensible objects for EC. It is not specific to any particular agent system. The GAglet package defines of set of EC extensions and utilities for IBM's Aglets. These packages are discussed in greater detail below. The ECbasics Package This package defines a number of key interfaces for the objects exchanged in EC interactions. These include: Randomizable This interface defines methods that allow one to generate a "random" instance of an object. What this means depends on the particular class that implements the interface. For instance, for a BooleanGene (see below), this interface implements methods that generate a gene with an equiprobable, random (true or false) value.
- 18. Mutatable This interface defines methods that cause a random change in an object. What this means depends on the particular class that implements this interface. For instance, in a BooleanGene (see below), this interface implements methods that change a true valued gene to a false valued gene, or vice versa. Gene This interface trivially extends Randomizable and Mutatable. Genotype This interface extends Randomizable and Mutatable, but also defines methods to generate objects that implement Sperm or Egg interfaces (see below). Gamete This interface defines methods that can "get" and "set" segments of an object. What this means depends on the class that implements this interface. Sperm This interface extends Gamete, but only trivially. Egg This interface extends Gamete, but also defines routines that can combine the implementing class with a class implementing the Sperm interface. The result is an object implementing the Genotype interface (i.e., the genotype of a potential child). Example Objects BooleanGene implements Gene This class is the typical Boolean gene often used in GAs. FloatGene implements Gene This class is a floating point gene, like those often used in evolutionary programming. Its mutation operator is a Gaussian "creep" around the current gene value. VectorGenotype extends Vector, implements Genotype This class is the typical chromosome string often used in GAs, but it also inherits all the functionality of the Java Vector class. VectorEgg and VectorSperm extend Vector, implement Egg or Sperm, respectively These classes implement the recombination of string chromosomes typically used in GAs. Other Possibilities Within the framework, Genes, Genotypes, Sperm, and Eggs can be any type of object. For instance, one could easily implement classes like • HashtableGenotype
- 19. • TreeGenotype • IntegerGene • VectorGene etc. The GAglet Package This package implements EC functionality for agents within the IBM Aglet framework. This key classes of this package are discussed below. There are two key utility classes: Plumage Objects from this class are used by agents to "advertise" themselves for mating to other agents. Phylum Objects from this class are used to initialize the basic genetic behaviors and intentions of a "first generation" agent, including its preferences in mates. There are also two key agent classes: GAglet extends Aglet This class is a basic Aglet agent, with methods that allow it to be initialized with either Phylum objects, or objects that implement the Genotype interface. Egglet extends GAglet This class implements methods that provide the agent with mate seeking behavior. These methods broadcast Plumage and Sperm objects, and call appropriate methods to create new child agents.
- 20. References Aglets Workbench. http://www.trl.ibm.co.jp/aglets/ Brooks, R.A., C. Breazeal (Ferrell), R. Irie, C. Kemp, M. Marjanovic, B. Scassellat and M. Williamson (1998). Alternate essences of intelligence. To appear in Proceedings of AAAI-98 Conference. AAAI Press. Deb, K. and Goldberg, D. E. (1989). An investigation of niche and species formation in genetic function optimization. Proceedings of the Third International Conference on Genetic Algorithms. p. 42--50. Franklin and Graesser (1997). Is it an agent, or just a program? : A taxonomy for autonomous agents. In Proceedings of the Third International Workshop on Agent Theories, Architectures, and Languages. Springer-Verlag. pp. 21-35. Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Addison-Wesley. Reading, MA Holland, J. H. (1975). Adaptation in natural and artificial systems. The University of Michigan Press. Ann Arbor, MI. Horn, J. and Goldberg, D. E. and Deb, K. (1994). Implicit niching in a learning classifier system: Nature's way. Evolutionary Computation, 2(1). p. 37-66. Kapsalis, A., Smith, G. D. and Rayward-Smith, V. J. (1994). A unified paradigm for parallel genetic algorithms. In T. Fogarty (ed.) Evolutionary Computing AISB Workshop. Springer-Verlag. 131-149. Koza, J. R. (1992). Genetic Programming: On The Programming of Computers By Means of Natural Selection. MIT Press. Ray, Thomas (1990). An approach to the synthesis of life. In Langton, C., Taylor, C. Farmer, J. D. and Rasmussen, S. (eds.) Artificial Life II. Addison-Wesley. p. 371-408 Rosin, C. D. and Belew, R. K. (1997). New methods in competitive coevolution. Evolutionary Computation. 5(1). pp. 1-29 Smith, R. E. (1995). Memory Exploitation in Learning Classifier Systems. Evolutionary Computation, 2(3). p. 199-220 Smith R. E. and Dike, B. A. (1995). Learning novel fighter combat maneuver rules via genetic algorithms. International Journal of Expert Systems. 8 (3). 247-276. Smith, R. E., Forrest, S. and Perelson, A. S. (1993). Searching for diverse, cooperative populations with genetic algorithms. Evolutionary Computation, 1(2). p. 127-149. Smith, R. E. and Goldberg, D. E. (1992). Diploidy and dominance in artificial genetic search. Complex Systems, 6(3). p. 251-285. Smith, R. E. and Taylor, N. (1998).A framework for evolutionary computation in agent- based systems. In C. Looney and J. Castaing (eds.) Proceedings of the 1998 International Conference on Intelligent Systems. ISCA Press. p. 221-224 Wooldridge and Jennings (1996). Software agents. IEE Review. January 1996. 17-20.