Evolving the Cooperative Behaviour in Unreal Bots
2 likes1,804 views
The document discusses the improvement of AI for Unreal bots using genetic algorithms, focusing on two approaches: co-evolution with different parameters per bot and a shared parameter approach for the team. Both methods outperform standard bots, with the shared parameter approach (cr-team) yielding the best results. Future work includes further experiments and enhancements to the AI's finite state machine.
Evolving the Cooperative Behaviour in Unreal Bots
- 1. CIG2010 A.M. Mora, M.A. Moreno, J.J. Merelo, P.A. Castillo, M.G. Arenas, J.L.J. Laredo Dpto. ATC University of Granada
- 3. CIG2010 It is a First Person Shooter (FPS): Very famous since 1998, because it had an excellent enemies’ AI, so it was (and is) a very good game for multiplayer battlematches between users and the so-called Bots (authonomous players).
- 4. CIG2010 It offers the Unreal Editor (included with the game): It is easy to create Mods:
- 5. CIG2010 There is an own language (UnrealScript). The editor is also an IDE, since it is possible to access the whole tree of classes, and write and compile new code inside it. But it has some flaws such as a limited size for arrays, its is slower than other languages, or the dificulty to debug the code.
- 6. CIG2010 It is based on a Finite State Machine: Example: State Roaming
- 7. CIG2010 A Bot decides what to do (changing between the states and substates) in the FSM depending on some parameters or thresholds. For instance: (in Roaming state) IF distance_to_enemy() < MIN_DIST_SHOOT THEN go_to_state(Attacking) In addition, there are some values used to calculate others, such as penalties, multiplication factors, etc.
- 8. CIG2010 Identify the ‘main’ Put them into set of parameters an array Evolve it with an Evolutionary Algorithm (Genetic Algorithm)
- 9. CIG2010 Created by Johann Dréo
- 10. CIG2010 Genetic Algorithm FITNESS EVALUATION based bot (GA-Bot) population Std AI Std Std AI AI GA EVOLUTIONARY PROCESS
- 11. CIG2010 Identify the ‘main’ Put them into set of parameters an array (for a Team) Evolve it with an Evolutionary Algorithm (Genetic Algorithm)
- 12. CIG2010 Genetic Algorithm FITNESS EVALUATION based team of bots (GT-Bot) Std Std AI AI Std AI population vs GA EVOLUTIONARY PROCESS OR
- 13. CIG2010 cr-Team All the bots in the team have the same chromosome. cr-Bot Every bot in the team has a different chromosome. A shape of co-evolution.
- 14. CIG2010 The set of thresholds and parameters to consider in a Team evolution is:
- 15. CIG2010 The real-coded GA we have used in this work considers: Chromosome Lentgh 40 Population size 30 Number of Generations 30 Simple Gene Mutation Mutation probability 0.01 Two point crossover Crossover probability 0.6 Time per chromosome 90 seconds
- 16. CIG2010 And the Fitness Function: Each team is evaluated after 90 seconds The fitness is a sumatory considering the bots’ results: - number of frags - times defeated - number of items collected - weapons collected - power of weapons
- 17. CIG2010 1 cr-Bot based Team (2 Bots) vs 3 standard 2-bots teams
- 18. CIG2010 1 cr-Team based Team (2 Bots) vs 3 standard 2-bots teams
- 19. CIG2010 1 cr-Team based Team (4 Bots) vs 3 standard 4-bots teams
- 20. CIG2010 1 cr-Bot based Team (2 Bots) vs 1 cr-Team based Team (2 bots) vs 2 standard 2-bots teams vs
- 21. CIG2010 We have improved the AI of Unreal Bots using a Genetic Algorithm, so they can behave better in a team. We have implemented 2 approaches: • one based in having a different set of parameters per bot (co- evolution) – cr-Bot • another based in sharing all the bots in the team the same set of parameters – cr-Team The results show that both approaches performs better than the standard bots and the cr-Team is the best of them. FUTURE WORK: - perform more experiments - improve the FSM of the AI for teams (sets of rules)
Editor's Notes
- #4: A game where you just can see the hand of your character. The objective is easy… kill the enemies before they kill you! Even in the Battlematch game where you fight against other players (human or bots)
- #5: In addition to the excellent AI. We started our researchings in 2000 and in that time it was the best environment (and the best game). We know there are some other possibilities, such as Pogamut…
- #6: IDE = Integrated Development Environment Unrealscript is a mixture between C and Java (quite similar to Java), but considering the so-called States (marked in the code as labels) Just can write in a file
- #7: Roaming -> the state where the bot moves around searching for items, enemies, etc States -> Roaming or Wandering This example were built by us (analysing the code or Unreal 1).
- #8: Some of them hard-coded.
- #9: It is very easy…
- #10: i initialization f(x) evaluation Re replacing Mu mutation ? end condition Cr cross Se selection X* best solution
- #12: It is very easy…
- #13: There are two different approaches: One considering the same chromosome for all the bots Other considering a different chromosome for every bot
- #14: There are two different approaches: One considering the same chromosome for all the bots Other considering a different chromosome for every bot
- #15: The most important ones. In addition there are some other parameters just useful for individuals.
- #16: It is a generational algorithm. Mutation add or substract a number in [0,1] Crossover choose two points and interchange the values between them
- #18: The teams of bots fight during 20 hours. Each bot with a different chromosome. The experiment has been performed 3 times in each map (and calculate an average). The cr-Bot team is better than the standard teams.
- #19: The teams of bots fight during 20 hours. All the bots with the same chromosome. The experiment has been performed 3 times in each map (and calculate an average). The cr-Team team is better than the standard teams.
- #20: The teams of bots fight during 20 hours. All the bots with the same chromosome. The experiment has been performed 3 times the map (and calculate an average). The cr-Team is better than the standard teams.
- #21: The teams of bots fight during 20 hours. All the bots with the same chromosome. The experiment has been performed 3 times the map (and calculate an average). The cr-Team is better than the cr-Bot.
- #22: We improved the FSM for one bot