Evolving Evil: Optimizing Flocking Strategies through Genetic Algorithms for the Ghost Team in the Game of Ms. PacMan

Evolving Evil: Optimizing Flocking
Strategies through Genetic
Algorithms for the Ghost Team in
the Game of Ms. PacMan
F. Liberatore
A. Mora
P. Castillo
J.J. Merelo

Outline
The Game of Ms. PacMan.
State of the Art.
Flocking.
Flocking Strategies for the Ghost
Team.
A Genetic Algorithm for Flocking
Strategies.
Experiments.
Conclusions.

What is this about?
In this work we propose:
A Controller for the Ghost
Team in the game of Ms. PacMan
…
… based on Flocking
Strategies (more about this in
the following) …
… optimized by a Genetic
Algorithm.

The game of Ms. PacMan
[1/2]
The game of Ms. PacMan:
Released on 1981.
Chosen for the Ms. PacMan vs
Ghosts competition.
Participants can submit
controllers for Ms.PacMan and
the Ghosts.
The objective of Ms. PacMan
is to maximize the final score.
The objective of the Ghosts is
to minimize it.

The game of Ms. PacMan
[2/2]
Movement restrictions for the
Ghosts:
A ghost can never stop.
A ghost can choose its
direction only at a junction.
A ghost cannot turn back on
itself.
At every tick of the game all
the ghosts obligatorily reverse
their direction according to a
small random probability.

State of the Art [1/2]
Most contributions proposed controllers for Ms.
PacMan:
Genetic Programming (Alhejali and Lucas 2010, 2011,
Brandstetter and Ahmadi 2012),
Genetic Algorithms for parameters optimization
(Thawonmas 2010),
Evolved Neural Networks (Burrow and Lucas 2009,
Keunhyun and SungBae 2010),
Ant Colonies (Martin et al. 2010),
Monte Carlo Tree Search (Samothrakis et al. 2011,
Ikehata and Ito 2011, Alhejali and Lucas 2013),
Reinforcement Learning (Bom et al. 2013).

State of the Art [2/2]
Controllers for the Ghosts:
Monte Carlo Tree Search (Nguyen and
Thawonmas 2011, 2013),
Influence Maps (Svensson and Johansson
2012),
Neural Networks (JiaYue et al. 2011),
Genetic Algorithm + Rules (Gagne and
Congdon 2012),
Competitive CoEvolution of Ms. PacMan and
Ghosts controllers (Cardona et al. 2013).

Flocking [1/2]
Swarm Intelligence technique:
Coordinated intelligence that arises from the
collective behavior of decentralized,
selforganized systems.
Flocking:
Developed to mimic lifelike behaviors of groups
of beings.
Flocking systems consist of a population of
simple agents (or boids) interacting locally with
one another depending on the distance between
them.

Flocking [2/2]
The agents follow very simple steering
behaviors:
∙ Separation: makes the agent steer away from close
flock mates.
∙ Alignment: makes the agent steer toward the
average heading of the flock.
∙ Cohesion: makes the agent steer toward the
average position of distant flock mates.
The interactions between such agents lead to
the emergence of "intelligent" global behavior

Flocking Strategies [1/4]
Generalization of the concept of Flocking.
The direction taken by a ghost at a junction
depends on:
∙ State:
Normal,
Vulnerable,
Flashing.
∙ Position WRT that of the other actors:
Ms. PacMan,
Powerpills,
Other Ghosts (depending on their state).

Each Ghost is surrounded by
concentric rings called
Neighborhoods.
The neighborhoods can have
different radius and size.
Each neighborhood defines
the strength of the
interaction between the ghost
and the other actors:
∙ Negative value Separation→
∙ Positive value Cohesion→

Formally, a Flocking Strategy is represented
by a tuple (N, δ, Α):
N, number of neighborhoods, indexed by n=1,…,N.
δ∈ℝN

, vector of N real positive numbers. Each
element δn
is associated to one neighborhood and
represents its maximum radius.
Α∈Μ, threedimensional matrix with real
entries. Each element αs,a,n
is the magnitude of
the steering force on the current ghost in state
s, resulting from the interaction with the actor
a, falling into neighborhood n.

First, all the interactions
with the other actors are
calculated as vectors.
Then, all the vectors are
summed to find the total
interaction vector.
Finally, the direction
taken by the ghost (UP, DOWN,
LEFT, RIGHT) depends on the
component of the total
interaction vector having
maximum absolute value.

A Genetic Algorithm for
Flocking Strategies could be designed manually
by an expert.
We propose a standard Genetic Algorithm for
the automatic definition of effective Ghosts
agents:
Each individual is a Flocking Strategy,
represented as a tuple (N, δ, Α).
Random initial population.
Roulette wheel selection.
Crossover: two parents generate one child by
random recombination of their chromosomes.

A Genetic Algorithm for
Fitness function:
∙ The objective is to create effective Ghosts that
perform well against different Ms. PacMan strategies.
∙ Each individual is tested against two Ms. PacMan
agents: StarterPacMan (SPM) and NearestPillPacMan
(NPPM).
∙ To ensure robustness in the face of random events,
the game is simulated 30 times for each Ms. PacMan
agent and 95% confidence intervals are calculated.
∙ Fitness is given by:
FITNESS = 1/CI+
SPM
+ 1/CI+
NPPM
Where CI+
represents the upper limit of the 95%
confidence interval.

Experiments [1/2]
Comparison of the performance of the Ghosts
controllers obtained with different values of
the parameter N (number of Neighborhoods)
N=1 N=2 N=3 N=4 N=5
Best
FITNESS1
783.38 726.84 815.66 766.96 720.20
Avg.
FITNESS1
871.30 ±
55.45
861.57 ±
70.13
876.31 ±
65.06
905.86 ±
62.66
863.17 ±
72.15
Avg. CPU
time (s)
1373 ±
150.66
1484.3 ±
122.01
1561 ±
193.94
1562.60
± 109.90
1473.00
± 74.02

Experiments [2/2]
Performances of the controllers included in
the competition framework.
Controller Aggressive
Ghosts
Legacy Legacy 2
The
Reckoning
Random
Ghosts
Starter
Ghosts
FITNESS1
1893.13 2210.9 1429.20 4200.70 1603.49

Conclusions [1/2]
Sample controller (link)
Flocking Strategies: sets of behavior rules that
determine the next move of an agent as a force
resulting from the interaction of the agents in
the game.
Genetic Algorithm presented to design optimized
strategies.
The fitness function evaluates each individual
by pitting it against two Ms. PacMan controllers
30 times, so as to avoid noise in the function.

Conclusions [2/2]
Future lines of research:
∙ Include in the fitness function the best Ms.
PacMan controllers that took part to the
competition.
∙ Compare the Flocking Strategybased
controller with the best Ghosts controllers that
took part to the competition.
∙ Explore optimization methods other than
Genetic Algorithms.
∙ Change the fitness function structure to
reduce the computation requirements.

Evolving Evil: Optimizing Flocking Strategies through Genetic Algorithms for the Ghost Team in the Game of Ms. PacMan

More Related Content

Viewers also liked (10)

More from Antonio Mora (20)

Recently uploaded (20)

Evolving Evil: Optimizing Flocking Strategies through Genetic Algorithms for the Ghost Team in the Game of Ms. PacMan