CoSECiVi'15 - An overview on the termination conditions in the evolution of game bots
0 likes400 views
The document provides an overview of research on termination conditions for evolving game bots, particularly in real-time strategy games using genetic programming. It discusses previous works, methodologies, experimental setups, and various termination criteria including generations, age of outliers, fitness thresholds, and replacement rates. The findings suggest that fitness thresholds yield the best scores, though challenges remain in identifying optimal values and ensuring completion within computational limits.
![Methodology: Termination Criteria
[NG] Number of Generations
➔ 30, 50, 100, 200 Generations
[AO] Age of Outliers
➔ 1, 1.5, 2, 2.5 times the IQR
[RT] Replacement Rate
➔ n/2, n/4, n/8, n/16 with n = N/2
[FT] Fitness Threshold
➔ 22, 24, 26, 28
[FI] Fitness Improvement
➔ 3, 7, 10, 15](https://image.slidesharecdn.com/cosecivi15-anoverviewontheterminationconditionsintheevolutionofgamebots-150628153145-lva1-app6892/85/CoSECiVi-15-An-overview-on-the-termination-conditions-in-the-evolution-of-game-bots-10-320.jpg)
![Conclusions I
[AO] Not good. As more restrictive
values don’t imply a significant score
improvement.
[FT] Best score. But the optimum fitness
might not be know. Needs the highest
amount of generations. It might not finish.
[FI] Useful to ‘detect’ local optima.
[RT] Provides the best results
considering all metrics.](https://image.slidesharecdn.com/cosecivi15-anoverviewontheterminationconditionsintheevolutionofgamebots-150628153145-lva1-app6892/85/CoSECiVi-15-An-overview-on-the-termination-conditions-in-the-evolution-of-game-bots-18-320.jpg)
CoSECiVi'15 - An overview on the termination conditions in the evolution of game bots
- 1. An overview on the termination conditions in the evolution of game bots A. Fernández-Ares, P. García-Sánchez, A.M. Mora, P.A. Castillo, J.J. Merelo, M.G. Arenas Universidad de Granada (Spain)
- 2. A summary of previous works Pablo García-Sánchez, Antonio Fernández-Ares, Antonio Miguel Mora, Pedro A. Castillo Valdivieso, Jesús González, Juan Julián Merelo Guervós: Tree Depth Influence in Genetic Programming for Generation of Competitive Agents for RTS Games. EvoApplications 2014: 411-421 Antonio Fernández-Ares, Pablo García-Sánchez, Antonio Miguel Mora, Pedro A. Castillo, Juan Julián Merelo Guervós: Designing competitive bots for a real time strategy game using genetic programming. CoSECivi 2014: 159-172 Antonio Fernández-Ares, Pablo García-Sánchez, Antonio M. Mora, Pedro A. Castillo Valdivieso, Juan Julián Merelo Guervós, Maria I. García Arenas, Gustavo Romero: It's Time to Stop: A Comparison of Termination Conditions in the Evolution of Game Bots. EvoApplications 2015: 355- 368
- 3. Index 1. Introduction a. RTS b. Planet Wars c. Genetic Programming 2. GPBot a. Conditions b. Actions 3. SCORE - Fitness 4. Methodology a. Termination Criteria 5. Experimental Setup 6. Results a. Score b. Generations c. Tests d. Benchmark e. Comparative 7. Conclusions 8. Future Work
- 4. Introduction: RTS Real-Time Strategy games (RTS-games) ● Resources ● Units ● Buildings Victory: Get all resources, kill all enemy units or destroy enemy buildings
- 5. Introduction: Planet Wars Buildings: Planets Units: Space ships Resources: Ships generated in every planet
- 6. Introduction: Genetic Programming Evolutionary Algorithm that evolves binary decision tree ➔ Internal nodes: Conditions ➔ Leafs: Actions Individual -> behavioural model (solution) Evaluation -> Playing a game against a rival and getting a score Adapted operators
- 7. GPBot: Conditions A logical expression composed by, at least, one extracted standard game state variable and a value between 0 and 1 ● myShipsEnemyRatio ● myShipsLandedFlyingRatio ● myPlanetsEnemyRatio ● myPlanetsTotalRatio ● actualMyShipsRatio ● actualLandedFlyingRatio
- 8. GPBot: Actions The possible actions just involve the movement of an amount of ships from a source to a destination planet ● Attack Nearest (Neutral|Enemy|NotMy)Planet ● Attack Weakest (Neutral|Enemy|NotMy) Planet ● Attack Wealthiest (Neutral|Enemy|NotMy) Planet ● Attack Beneficial (Neutral|Enemy|NotMy) Planet ● Attack Quickest (Neutral|Enemy|NotMy) Planet} ● Attack (Neutral|Enemy|NotMy) Base ● Attack Random Planet ● Reinforce Nearest Planet ● Reinforce Base ● Reinforce Wealthiest Planet
- 9. Score (Fitness) SCORE BASED IN TURNS IN VICTORIES SCORE BASED IN TURNS IN DEFEATS SCORE BASED IN VICTORIES > >
- 10. Methodology: Termination Criteria [NG] Number of Generations ➔ 30, 50, 100, 200 Generations [AO] Age of Outliers ➔ 1, 1.5, 2, 2.5 times the IQR [RT] Replacement Rate ➔ n/2, n/4, n/8, n/16 with n = N/2 [FT] Fitness Threshold ➔ 22, 24, 26, 28 [FI] Fitness Improvement ➔ 3, 7, 10, 15
- 11. Experimental Setup Parameter Name Value Population Size (N) 32 Number of battles for scoring (NB) 30 Re-evaluation of individuals Yes Crossover type Sub-tree crossover Crossover rate 0.5 Mutation 1-node mutation Mutation rate 0.25 Selection 2-tournament Replacement Generational Maximum Tree Depth 7 Runs 36
- 12. SCORE of the best individuals (of all runs) grouped by criterion
- 13. GENERATION attained (of all runs) grouped by criterion
- 14. Kruskal-Wallis Test of samples of each criterion by SCORE an GENERATIONS. Black means no statistically significant difference has been found.
- 15. Percentage of victories (of the 36 best individuals) in benchmark against five different competitive bots available in the literature in 100 maps
- 16. Linear regression of the SCORE (fitness) with the results of the benchmark (Percentage of victories)
- 17. Average score of the best individual and average reached generations per termination criteria Average results of every criterion (relative with respect to NG_30)
- 18. Conclusions I [AO] Not good. As more restrictive values don’t imply a significant score improvement. [FT] Best score. But the optimum fitness might not be know. Needs the highest amount of generations. It might not finish. [FI] Useful to ‘detect’ local optima. [RT] Provides the best results considering all metrics.
- 19. Conclusions II Fitness Threshold would be the most desirable option ➔ It attains the best score. ➔ It is quite difficult to find an optimal fitness value to use (normally it is unknown). ➔ Requires more computational budget, and it is possible that it never ends (the criterion is not met). Replacement Rate as stopping criterion, since it is a compromise solution which relies in the population improvement without an implicit use of the fitness.
- 20. Future work ● New problems (and algorithms) are being addressed. ○ P. García-Sánchez, Alberto Tonda, Giovanni Squillero and JJ. Merelo: Towards Automatic StarCraft Strategy Generation Using Genetic Programming. Accepted at CIG 2015. ● Mechanisms to improve the EA. ● Use larger and complex decision trees.
- 21. Questions Thank you! Antonio Fernández-Ares antares@ugr.es - @antaress Antonio M. Mora amorag@ugr.es - @amoragar Pablo García-Sánchez pablogarcia@ugr.es - @fergunet