SlideShare a Scribd company logo
Genetic Algorithms
Genetic Algorithms
Muhannad Harrim
Muhannad Harrim
Introduction
Introduction
 After scientists became disillusioned with
After scientists became disillusioned with
classical and neo-classical attempts at
classical and neo-classical attempts at
modeling intelligence, they looked in other
modeling intelligence, they looked in other
directions.
directions.
 Two prominent fields arose, connectionism
Two prominent fields arose, connectionism
(neural networking, parallel processing) and
(neural networking, parallel processing) and
evolutionary computing.
evolutionary computing.
 It is the latter that this essay deals with -
It is the latter that this essay deals with -
genetic algorithms and genetic programming.
genetic algorithms and genetic programming.
What is GA
What is GA

A
A genetic algorithm
genetic algorithm (or
(or GA
GA) is a search technique
) is a search technique
used in computing to find true or approximate
used in computing to find true or approximate
solutions to optimization and search problems.
solutions to optimization and search problems.
 Genetic algorithms are categorized as global search
Genetic algorithms are categorized as global search
heuristics.
heuristics.

Genetic algorithms are a particular class of
Genetic algorithms are a particular class of
evolutionary algorithms that use techniques inspired
evolutionary algorithms that use techniques inspired
by evolutionary biology such as inheritance,
by evolutionary biology such as inheritance,
mutation, selection, and crossover (also called
mutation, selection, and crossover (also called
recombination).
recombination).
What is GA
What is GA
 Genetic algorithms are implemented as a computer
Genetic algorithms are implemented as a computer
simulation in which a population of abstract
simulation in which a population of abstract
representations (called chromosomes or the genotype
representations (called chromosomes or the genotype
or the genome) of candidate solutions (called
or the genome) of candidate solutions (called
individuals, creatures, or phenotypes) to an
individuals, creatures, or phenotypes) to an
optimization problem evolves toward better solutions.
optimization problem evolves toward better solutions.
 Traditionally, solutions are represented in binary as
Traditionally, solutions are represented in binary as
strings of 0s and 1s, but other encodings are also
strings of 0s and 1s, but other encodings are also
possible.
possible.
What is GA
What is GA

The evolution usually starts from a population of
The evolution usually starts from a population of
randomly generated individuals and happens in
randomly generated individuals and happens in
generations.
generations.

In each generation, the fitness of every individual in
In each generation, the fitness of every individual in
the population is evaluated, multiple individuals are
the population is evaluated, multiple individuals are
selected from the current population (based on their
selected from the current population (based on their
fitness), and modified (recombined and possibly
fitness), and modified (recombined and possibly
mutated) to form a new population.
mutated) to form a new population.
What is GA
What is GA
 The new population is then used in the next
The new population is then used in the next
iteration of the algorithm.
iteration of the algorithm.
 Commonly, the algorithm terminates when
Commonly, the algorithm terminates when
either a maximum number of generations has
either a maximum number of generations has
been produced, or a satisfactory fitness level
been produced, or a satisfactory fitness level
has been reached for the population.
has been reached for the population.
 If the algorithm has terminated due to a
If the algorithm has terminated due to a
maximum number of generations, a
maximum number of generations, a
satisfactory solution may or may not have
satisfactory solution may or may not have
been reached.
been reached.
Key terms
Key terms
 Individual
Individual - Any possible solution
- Any possible solution
 Population
Population - Group of all
- Group of all individuals
individuals
 Search Space
Search Space - All possible solutions to the problem
- All possible solutions to the problem
 Chromosome
Chromosome - Blueprint for an
- Blueprint for an individual
individual
 Trait
Trait - Possible aspect (
- Possible aspect (features)
features) of an
of an individual
individual
 Allele -
- Possible settings of trait (black, blond, etc.)
 Locus
Locus - The position of a
- The position of a gene
gene on the
on the chromosome
chromosome
 Genome
Genome - Collection of all
- Collection of all chromosomes
chromosomes for an
for an
individual
individual
Chromosome, Genes and
Genomes
Genotype and Phenotype
 Genotype:
– Particular set of genes in a genome
 Phenotype:
– Physical characteristic of the genotype
(smart, beautiful, healthy, etc.)
Genotype and Phenotype
GA Requirements
GA Requirements

A typical genetic algorithm requires two things to be defined:
A typical genetic algorithm requires two things to be defined:

a genetic representation of the solution domain, and
a genetic representation of the solution domain, and

a fitness function to evaluate the solution domain.
a fitness function to evaluate the solution domain.

A standard representation of the solution is as an array of bits.
A standard representation of the solution is as an array of bits.
Arrays of other types and structures can be used in essentially
Arrays of other types and structures can be used in essentially
the same way.
the same way.

The main property that makes these genetic representations
The main property that makes these genetic representations
convenient is that their parts are easily aligned due to their
convenient is that their parts are easily aligned due to their
fixed size, that facilitates simple crossover operation.
fixed size, that facilitates simple crossover operation.

Variable length representations may also be used, but
Variable length representations may also be used, but
crossover implementation is more complex in this case.
crossover implementation is more complex in this case.

Tree-like representations are explored in Genetic
Tree-like representations are explored in Genetic
programming.
programming.
Representation
Representation
Chromosomes could be:
Chromosomes could be:

Bit strings (0101 ... 1100)
Bit strings (0101 ... 1100)
 Real numbers (43.2 -33.1 ... 0.0 89.2)
Real numbers (43.2 -33.1 ... 0.0 89.2)
 Permutations of element (E11 E3 E7 ... E1 E15)
Permutations of element (E11 E3 E7 ... E1 E15)

Lists of rules (R1 R2 R3 ... R22 R23)
Lists of rules (R1 R2 R3 ... R22 R23)
 Program elements (genetic programming)
Program elements (genetic programming)
 ... any data structure ...
... any data structure ...
GA Requirements
GA Requirements

The fitness function is defined over the genetic representation
The fitness function is defined over the genetic representation
and measures the
and measures the quality
quality of the represented solution.
of the represented solution.

The fitness function is always problem dependent.
The fitness function is always problem dependent.

For instance, in the
For instance, in the knapsack problem we want to maximize
we want to maximize
the total value of objects that we can put in a knapsack of
the total value of objects that we can put in a knapsack of
some fixed capacity.
some fixed capacity.

A representation of a solution might be an array of bits, where
A representation of a solution might be an array of bits, where
each bit represents a different object, and the value of the bit
each bit represents a different object, and the value of the bit
(0 or 1) represents whether or not the object is in the knapsack.
(0 or 1) represents whether or not the object is in the knapsack.

Not every such representation is valid, as the size of objects
Not every such representation is valid, as the size of objects
may exceed the capacity of the knapsack.
may exceed the capacity of the knapsack.

The
The fitness
fitness of the solution is the sum of values of all objects in
of the solution is the sum of values of all objects in
the knapsack if the representation is valid, or 0 otherwise. In
the knapsack if the representation is valid, or 0 otherwise. In
some problems, it is hard or even impossible to define the
some problems, it is hard or even impossible to define the
fitness expression; in these cases, interactive genetic
fitness expression; in these cases, interactive genetic
algorithms are used.
algorithms are used.
A fitness function
A fitness function
Basics of GA
Basics of GA

The most common type of genetic algorithm works like this:
The most common type of genetic algorithm works like this:

a population is created with a group of individuals created
a population is created with a group of individuals created
randomly.
randomly.

The individuals in the population are then evaluated.
The individuals in the population are then evaluated.

The evaluation function is provided by the programmer and
The evaluation function is provided by the programmer and
gives the individuals a score based on how well they perform
gives the individuals a score based on how well they perform
at the given task.
at the given task.

Two individuals are then selected based on their fitness, the
Two individuals are then selected based on their fitness, the
higher the fitness, the higher the chance of being selected.
higher the fitness, the higher the chance of being selected.

These individuals then "reproduce" to create one or more
These individuals then "reproduce" to create one or more
offspring, after which the offspring are mutated randomly.
offspring, after which the offspring are mutated randomly.

This continues until a suitable solution has been found or a
This continues until a suitable solution has been found or a
certain number of generations have passed, depending on the
certain number of generations have passed, depending on the
needs of the programmer.
needs of the programmer.
General Algorithm for GA
General Algorithm for GA
 Initialization
Initialization
 Initially many individual solutions are randomly
Initially many individual solutions are randomly
generated to form an initial population. The
generated to form an initial population. The
population size depends on the nature of the problem,
population size depends on the nature of the problem,
but typically contains several hundreds or thousands
but typically contains several hundreds or thousands
of possible solutions.
of possible solutions.
 Traditionally, the population is generated randomly,
Traditionally, the population is generated randomly,
covering the entire range of possible solutions (the
covering the entire range of possible solutions (the
search space
search space).
).
 Occasionally, the solutions may be "seeded" in areas
Occasionally, the solutions may be "seeded" in areas
where optimal solutions are likely to be found.
where optimal solutions are likely to be found.
General Algorithm for GA
General Algorithm for GA

Selection
Selection

During each successive generation, a proportion of the
During each successive generation, a proportion of the
existing population is selected to breed a new generation.
existing population is selected to breed a new generation.

Individual solutions are selected through a
Individual solutions are selected through a fitness-based
fitness-based
process, where fitter solutions (as measured by a fitness
process, where fitter solutions (as measured by a fitness
function) are typically more likely to be selected.
function) are typically more likely to be selected.

Certain selection methods rate the fitness of each solution and
Certain selection methods rate the fitness of each solution and
preferentially select the best solutions. Other methods rate
preferentially select the best solutions. Other methods rate
only a random sample of the population, as this process may
only a random sample of the population, as this process may
be very time-consuming.
be very time-consuming.

Most functions are stochastic and designed so that a small
Most functions are stochastic and designed so that a small
proportion of less fit solutions are selected. This helps keep
proportion of less fit solutions are selected. This helps keep
the diversity of the population large, preventing premature
the diversity of the population large, preventing premature
convergence on poor solutions. Popular and well-studied
convergence on poor solutions. Popular and well-studied
selection methods include roulette wheel selection and
selection methods include roulette wheel selection and
tournament selection.
tournament selection.
General Algorithm for GA
General Algorithm for GA
 In roulette wheel selection, individuals are
In roulette wheel selection, individuals are
given a probability of being selected that is
given a probability of being selected that is
directly proportionate to their fitness.
directly proportionate to their fitness.

Two individuals are then chosen randomly
Two individuals are then chosen randomly
based on these probabilities and produce
based on these probabilities and produce
offspring.
offspring.
General Algorithm for GA
General Algorithm for GA
Roulette Wheel’s Selection Pseudo Code:
Roulette Wheel’s Selection Pseudo Code:
for all members of population
for all members of population
sum += fitness of this individual
sum += fitness of this individual
end for
end for
for all members of population
for all members of population
probability = sum of probabilities + (fitness / sum)
probability = sum of probabilities + (fitness / sum)
sum of probabilities += probability
sum of probabilities += probability
end for
end for
loop until new population is full
loop until new population is full
do this twice
do this twice
number = Random between 0 and 1
number = Random between 0 and 1
for all members of population
for all members of population
if number > probability but less than next probability then
if number > probability but less than next probability then
you have been selected
you have been selected
end for
end for
end
end
create offspring
create offspring
end loop
end loop
General Algorithm for GA
General Algorithm for GA
 Reproduction
Reproduction
 The next step is to generate a second generation population of
The next step is to generate a second generation population of
solutions from those selected through genetic operators:
solutions from those selected through genetic operators:
crossover (also called recombination), and/or mutation.
crossover (also called recombination), and/or mutation.
 For each new solution to be produced, a pair of "parent"
For each new solution to be produced, a pair of "parent"
solutions is selected for breeding from the pool selected
solutions is selected for breeding from the pool selected
previously.
previously.
 By producing a "child" solution using the above methods of
By producing a "child" solution using the above methods of
crossover and mutation, a new solution is created which
crossover and mutation, a new solution is created which
typically shares many of the characteristics of its "parents".
typically shares many of the characteristics of its "parents".
New parents are selected for each child, and the process
New parents are selected for each child, and the process
continues until a new population of solutions of appropriate
continues until a new population of solutions of appropriate
size is generated.
size is generated.
General Algorithm for GA
General Algorithm for GA
 These processes ultimately result in the next
These processes ultimately result in the next
generation population of chromosomes that is
generation population of chromosomes that is
different from the initial generation.
different from the initial generation.
 Generally the average fitness will have
Generally the average fitness will have
increased by this procedure for the population,
increased by this procedure for the population,
since only the best organisms from the first
since only the best organisms from the first
generation are selected for breeding, along
generation are selected for breeding, along
with a small proportion of less fit solutions, for
with a small proportion of less fit solutions, for
reasons already mentioned above.
reasons already mentioned above.
Crossover
Crossover

the most common type is single point crossover. In single
the most common type is single point crossover. In single
point crossover, you choose a locus at which you swap the
point crossover, you choose a locus at which you swap the
remaining alleles from on parent to the other. This is complex
remaining alleles from on parent to the other. This is complex
and is best understood visually.
and is best understood visually.

As you can see, the children take one section of the
As you can see, the children take one section of the
chromosome from each parent.
chromosome from each parent.

The point at which the chromosome is broken depends on the
The point at which the chromosome is broken depends on the
randomly selected crossover point.
randomly selected crossover point.

This particular method is called single point crossover because
This particular method is called single point crossover because
only one crossover point exists. Sometimes only child 1 or
only one crossover point exists. Sometimes only child 1 or
child 2 is created, but oftentimes both offspring are created
child 2 is created, but oftentimes both offspring are created
and put into the new population.
and put into the new population.

Crossover does not always occur, however. Sometimes, based
Crossover does not always occur, however. Sometimes, based
on a set probability, no crossover occurs and the parents are
on a set probability, no crossover occurs and the parents are
copied directly to the new population. The probability of
copied directly to the new population. The probability of
crossover occurring is usually 60% to 70%.
crossover occurring is usually 60% to 70%.
Crossover
Crossover
Mutation
Mutation

After selection and crossover, you now have a new population
After selection and crossover, you now have a new population
full of individuals.
full of individuals.

Some are directly copied, and others are produced by
Some are directly copied, and others are produced by
crossover.
crossover.

In order to ensure that the individuals are not all exactly the
In order to ensure that the individuals are not all exactly the
same, you allow for a small chance of mutation.
same, you allow for a small chance of mutation.

You loop through all the alleles of all the individuals, and if
You loop through all the alleles of all the individuals, and if
that allele is selected for mutation, you can either change it by
that allele is selected for mutation, you can either change it by
a small amount or replace it with a new value. The probability
a small amount or replace it with a new value. The probability
of mutation is usually between 1 and 2 tenths of a percent.
of mutation is usually between 1 and 2 tenths of a percent.

Mutation is fairly simple. You just change the selected alleles
Mutation is fairly simple. You just change the selected alleles
based on what you feel is necessary and move on. Mutation is,
based on what you feel is necessary and move on. Mutation is,
however, vital to ensuring genetic diversity within the
however, vital to ensuring genetic diversity within the
population.
population.
Mutation
Mutation
General Algorithm for GA
General Algorithm for GA
 Termination
Termination
 This generational process is repeated until a
This generational process is repeated until a
termination condition has been reached.
termination condition has been reached.
 Common terminating conditions are:
Common terminating conditions are:
 A solution is found that satisfies minimum criteria
A solution is found that satisfies minimum criteria
 Fixed number of generations reached
Fixed number of generations reached
 Allocated budget (computation time/money) reached
Allocated budget (computation time/money) reached
 The highest ranking solution's fitness is reaching or has
The highest ranking solution's fitness is reaching or has
reached a plateau such that successive iterations no longer
reached a plateau such that successive iterations no longer
produce better results
produce better results
 Manual inspection
Manual inspection
 Any Combinations of the above
Any Combinations of the above
GA Pseudo-code
GA Pseudo-code
Choose initial population
Choose initial population
Evaluate the fitness of each individual in the population
Evaluate the fitness of each individual in the population
Repeat
Repeat
Select best-ranking individuals to reproduce
Select best-ranking individuals to reproduce
Breed new generation through crossover and mutation (genetic
Breed new generation through crossover and mutation (genetic
operations) and give birth to offspring
operations) and give birth to offspring
Evaluate the individual fitnesses of the offspring
Evaluate the individual fitnesses of the offspring
Replace worst ranked part of population with offspring
Replace worst ranked part of population with offspring
Until <terminating condition>
Until <terminating condition>
Symbolic AI VS. Genetic
Symbolic AI VS. Genetic
Algorithms
Algorithms

Most symbolic AI systems are very static.
Most symbolic AI systems are very static.

Most of them can usually only solve one given
Most of them can usually only solve one given
specific problem, since their architecture was
specific problem, since their architecture was
designed for whatever that specific problem was in
designed for whatever that specific problem was in
the first place.
the first place.

Thus, if the given problem were somehow to be
Thus, if the given problem were somehow to be
changed, these systems could have a hard time
changed, these systems could have a hard time
adapting to them, since the algorithm that would
adapting to them, since the algorithm that would
originally arrive to the solution may be either
originally arrive to the solution may be either
incorrect or less efficient.
incorrect or less efficient.

Genetic algorithms (or GA) were created to combat
Genetic algorithms (or GA) were created to combat
these problems; they are basically algorithms based
these problems; they are basically algorithms based
on natural biological evolution.
on natural biological evolution.
Symbolic AI VS. Genetic
Symbolic AI VS. Genetic
Algorithms
Algorithms

The architecture of systems that implement genetic algorithms
The architecture of systems that implement genetic algorithms
(or GA) are more able to adapt to a wide range of problems.
(or GA) are more able to adapt to a wide range of problems.

A GA functions by generating a large set of possible solutions
A GA functions by generating a large set of possible solutions
to a given problem.
to a given problem.

It then evaluates each of those solutions, and decides on a
It then evaluates each of those solutions, and decides on a
"fitness level" (you may recall the phrase: "survival of the
"fitness level" (you may recall the phrase: "survival of the
fittest") for each solution set.
fittest") for each solution set.

These solutions then breed new solutions.
These solutions then breed new solutions.

The parent solutions that were more "fit" are more likely to
The parent solutions that were more "fit" are more likely to
reproduce, while those that were less "fit" are more unlikely to
reproduce, while those that were less "fit" are more unlikely to
do so.
do so.

In essence, solutions are evolved over time. This way you
In essence, solutions are evolved over time. This way you
evolve your search space scope to a point where you can find
evolve your search space scope to a point where you can find
the solution.
the solution.

Genetic algorithms can be incredibly efficient if programmed
Genetic algorithms can be incredibly efficient if programmed
correctly.
correctly.
Genetic Programming
Genetic Programming
 In programming languages such as LISP, the mathematical
In programming languages such as LISP, the mathematical
notation is not written in standard notation, but in prefix
notation is not written in standard notation, but in prefix
notation. Some examples of this:
notation. Some examples of this:
 + 2 1
+ 2 1 :
: 2 + 1
2 + 1
 * + 2 1 2
* + 2 1 2 :
: 2 * (2+1)
2 * (2+1)
 * + - 2 1 4 9 :
* + - 2 1 4 9 : 9 * ((2 - 1) + 4)
9 * ((2 - 1) + 4)
 Notice the difference between the left-hand side to the right?
Notice the difference between the left-hand side to the right?
Apart from the order being different, no parenthesis! The
Apart from the order being different, no parenthesis! The
prefix method makes it a lot easier for programmers and
prefix method makes it a lot easier for programmers and
compilers alike, because order precedence is not an issue.
compilers alike, because order precedence is not an issue.
 You can build expression trees out of these strings that then
You can build expression trees out of these strings that then
can be easily evaluated, for example, here are the trees for the
can be easily evaluated, for example, here are the trees for the
above three expressions.
above three expressions.
Genetic Programming
Genetic Programming
Genetic Programming
Genetic Programming
 You can see how expression evaluation is thus a lot
You can see how expression evaluation is thus a lot
easier.
easier.
 What this have to do with GAs? If for example you
What this have to do with GAs? If for example you
have numerical data and 'answers', but no expression
have numerical data and 'answers', but no expression
to conjoin the data with the answers.
to conjoin the data with the answers.
 A genetic algorithm can be used to 'evolve' an
A genetic algorithm can be used to 'evolve' an
expression tree to create a very close fit to the data.
expression tree to create a very close fit to the data.
 By 'splicing' and 'grafting' the trees and evaluating
By 'splicing' and 'grafting' the trees and evaluating
the resulting expression with the data and testing it to
the resulting expression with the data and testing it to
the answers, the fitness function can return how close
the answers, the fitness function can return how close
the expression is.
the expression is.
Genetic Programming
Genetic Programming
 The limitations of genetic programming lie in
The limitations of genetic programming lie in
the
the huge
huge search space the GAs have to search
search space the GAs have to search
for - an infinite number of equations.
for - an infinite number of equations.
 Therefore, normally before running a GA to
Therefore, normally before running a GA to
search for an equation, the user tells the
search for an equation, the user tells the
program which operators and numerical ranges
program which operators and numerical ranges
to search under.
to search under.
 Uses of genetic programming can lie in stock
Uses of genetic programming can lie in stock
market prediction, advanced mathematics and
market prediction, advanced mathematics and
military applications .
military applications .
Evolving Neural Networks
Evolving Neural Networks
 Evolving the architecture of neural network is
Evolving the architecture of neural network is
slightly more complicated, and there have
slightly more complicated, and there have
been several ways of doing it. For small nets, a
been several ways of doing it. For small nets, a
simple matrix represents which neuron
simple matrix represents which neuron
connects which, and then this matrix is, in
connects which, and then this matrix is, in
turn, converted into the necessary 'genes', and
turn, converted into the necessary 'genes', and
various combinations of these are evolved.
various combinations of these are evolved.
Evolving Neural Networks
Evolving Neural Networks
 Many would think that a learning function could be
Many would think that a learning function could be
evolved via genetic programming. Unfortunately,
evolved via genetic programming. Unfortunately,
genetic programming combined with neural networks
genetic programming combined with neural networks
could be
could be incredibly
incredibly slow, thus impractical.
slow, thus impractical.
 As with many problems, you have to constrain what
As with many problems, you have to constrain what
you are attempting to create.
you are attempting to create.
 For example, in 1990, David Chalmers attempted to
For example, in 1990, David Chalmers attempted to
evolve a function as good as the delta rule.
evolve a function as good as the delta rule.
 He did this by creating a general equation based upon
He did this by creating a general equation based upon
the delta rule with 8 unknowns, which the genetic
the delta rule with 8 unknowns, which the genetic
algorithm then evolved.
algorithm then evolved.
Other Areas
Other Areas
 Genetic Algorithms can be applied to virtually any problem
Genetic Algorithms can be applied to virtually any problem
that has a large search space.
that has a large search space.
 Al Biles uses genetic algorithms to filter out 'good' and 'bad'
Al Biles uses genetic algorithms to filter out 'good' and 'bad'
riffs for jazz improvisation.
riffs for jazz improvisation.
 The military uses GAs to evolve equations to differentiate
The military uses GAs to evolve equations to differentiate
between different radar returns.
between different radar returns.
 Stock companies use GA-powered programs to predict the
Stock companies use GA-powered programs to predict the
stock market.
stock market.
Example
Example

f(x) = {MAX(x
f(x) = {MAX(x2
2
): 0 <= x <= 32 }
): 0 <= x <= 32 }

Encode Solution: Just use 5 bits (1 or 0).
Encode Solution: Just use 5 bits (1 or 0).

Generate initial population.
Generate initial population.
 Evaluate each solution against objective.
Evaluate each solution against objective.
Sol.
Sol. String
String Fitness
Fitness % of Total
% of Total
A
A 01101
01101 169
169 14.4
14.4
B
B 11000
11000 576
576 49.2
49.2
C
C 01000
01000 64
64 5.5
5.5
D
D 10011
10011 361
361 30.9
30.9
A
A 0
0 1
1 1
1 0
0 1
1
B
B 1
1 1
1 0
0 0
0 0
0
C
C 0
0 1
1 0
0 0
0 0
0
D
D 1
1 0
0 0
0 1
1 1
1
Example Cont’d
Example Cont’d

Create next generation of solutions
Create next generation of solutions

Probability of “being a parent” depends on the fitness.
Probability of “being a parent” depends on the fitness.

Ways for parents to create next generation
Ways for parents to create next generation

Reproduction
Reproduction

Use a string again unmodified.
Use a string again unmodified.

Crossover
Crossover

Cut and paste portions of one string to another.
Cut and paste portions of one string to another.

Mutation
Mutation

Randomly flip a bit.
Randomly flip a bit.

COMBINATION of all of the above.
COMBINATION of all of the above.
Checkboard example
 We are given an n by n checkboard in which every field
can have a different colour from a set of four colors.
 Goal is to achieve a checkboard in a way that there are no
neighbours with the same color (not diagonal)
1 2 3 4 5 6 7 8 9 10
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10
1
2
3
4
5
6
7
8
9
10
Checkboard example Cont’d
 Chromosomes represent the way the checkboard is colored.
 Chromosomes are not represented by bitstrings but by
bitmatrices
 The bits in the bitmatrix can have one of the four values 0,
1, 2 or 3, depending on the color.
 Crossing-over involves matrix manipulation instead of
point wise operating.
 Crossing-over can be combining the parential matrices in a
horizontal, vertical, triangular or square way.
 Mutation remains bitwise changing bits in either one
of the other numbers.
Checkboard example Cont’d
• This problem can be seen as a graph with n nodes
and (n-1) edges, so the fitness f(x) is defined
as:
f(x) = 2 · (n-1) ·n
Checkboard example Cont’d
• Fitnesscurves for different cross-over rules:
0 100 200 300 400 500
130
140
150
160
170
180
Fitness
Lower-Triangular Crossing Over
0 200 400 600 800
130
140
150
160
170
180
Square Crossing Over
0 200 400 600 800
130
140
150
160
170
180
Generations
Fitness
Horizontal Cutting Crossing Over
0 500 1000 1500
130
140
150
160
170
180
Generations
Verical Cutting Crossing Over
Questions
Questions
??
??
THANK YOU
THANK YOU

More Related Content

PPT
Genetic-Algorithms for engineering appl.ppt
PPT
Genetic algorithms
PPT
Genetic algorithms full lecture
PPT
Genetic algorithms
PDF
RM 701 Genetic Algorithm and Fuzzy Logic lecture
PDF
Genetic Algorithms in Artificial Intelligence
PPT
AI_PPT_Genetic-Algorithms.ppt
PPT
Genetic-Algorithms forv artificial .ppt
Genetic-Algorithms for engineering appl.ppt
Genetic algorithms
Genetic algorithms full lecture
Genetic algorithms
RM 701 Genetic Algorithm and Fuzzy Logic lecture
Genetic Algorithms in Artificial Intelligence
AI_PPT_Genetic-Algorithms.ppt
Genetic-Algorithms forv artificial .ppt

Similar to Genetic-Algorithms SDSDa SDD dfsAFF fsaf (20)

PPT
Genetic-Algorithms.ppt
PPT
Genetic-Algorithms.ppt
PPT
Genetic-Algorithms-computersciencepptnew.ppt
PPT
Genetic-Algorithms for machine learning and ai.ppt
PPT
4.Genetic-Algorithms.ppt
PPTX
Explanation and example of genetic algorithm
PDF
Data Science - Part XIV - Genetic Algorithms
PDF
F043046054
PDF
F043046054
PDF
F043046054
PDF
A Review On Genetic Algorithm And Its Applications
PPTX
Genetic algorithms
PPTX
Genetic algorithms
PDF
Parallel evolutionary approach paper
PPTX
Genetic Algorithms.pptx on dsa , algorithms , genetic algos
PDF
generic optimization techniques lecture slides
PPTX
Genetic Algorithm 2 -.pptx
PDF
PDF
Genetic algorithm
PDF
Soft Computing- Dr. H.s. Hota 28.08.14.pdf
Genetic-Algorithms.ppt
Genetic-Algorithms.ppt
Genetic-Algorithms-computersciencepptnew.ppt
Genetic-Algorithms for machine learning and ai.ppt
4.Genetic-Algorithms.ppt
Explanation and example of genetic algorithm
Data Science - Part XIV - Genetic Algorithms
F043046054
F043046054
F043046054
A Review On Genetic Algorithm And Its Applications
Genetic algorithms
Genetic algorithms
Parallel evolutionary approach paper
Genetic Algorithms.pptx on dsa , algorithms , genetic algos
generic optimization techniques lecture slides
Genetic Algorithm 2 -.pptx
Genetic algorithm
Soft Computing- Dr. H.s. Hota 28.08.14.pdf
Ad

More from dipesh257290 (7)

PPT
S_TSB_DPK ASDR RRE RTET TRTRT T TRTTTTTTTTTTT
PPT
HMSuz dasdfsa dfdgG DGFGFSD SDFGSDFA DFGSG SGDgg
PPT
ML55_Reactive power_SDSADSA SDasdas sadaFD ASDF
PPT
Load Cells in engineering dfsdfsdf sdfsdf fdfdf
PDF
IEI News December 2020.pdf
PPTX
DPC-6.ppt.pptx
PPT
hpfc_seminar_01.ppt
S_TSB_DPK ASDR RRE RTET TRTRT T TRTTTTTTTTTTT
HMSuz dasdfsa dfdgG DGFGFSD SDFGSDFA DFGSG SGDgg
ML55_Reactive power_SDSADSA SDasdas sadaFD ASDF
Load Cells in engineering dfsdfsdf sdfsdf fdfdf
IEI News December 2020.pdf
DPC-6.ppt.pptx
hpfc_seminar_01.ppt
Ad

Recently uploaded (20)

DOCX
Breast Pump Accessories Guide_ What You Need.docx
PDF
Wendy’s Menu Canada – Complete Guide 2025
PDF
How Food Data Scraping Is Revolutionizing Restaurant Growth Strategies
PPTX
06. 1649335909782_CLI 250 NUTRITION_2022.pptx
PDF
08_Mango_Dis_PARTIALSTEMPARASITE.pdf -farmers
PPTX
Lecture 2 Effect of water on shelf life of food.pptx
PPTX
SUSTAINABLE FOOD PRODUCTION and supply chain
PPTX
หลักสูตร Standard Barista for IPC barista
PDF
HealthyIndianBites:Eat Right, Live Right.pdf
PPTX
1. CLEAN AND MAINTAIN KITCHEN PREMISES.pptx
PDF
Top 10 Viral Food Menus in 2025 | Menu Makanan Hits yang Lagi Viral!** **Must...
PDF
Food in the Maldives and Its Unique Culinary Tradition
DOCX
ZDSPGC-PART-TIME-LOADING.docx and faculty
PDF
Brown-Illustrative-Abstract-Group-Project-Presentation-1.pdf
PPTX
role of SGLT2s in Non-Diabetic Patients.pptx
PPTX
FST-401 lecture # 12 food chemistry.pptx
PDF
Agricultural_Census_Main_Findings_Report.pdf
DOC
Aber毕业证学历认证,西蒙弗雷泽大学毕业证留学学历
PPTX
Agrisphere ai powered presision farming marketplace
PPT
Food Chain and Food Web in the world.ppt
Breast Pump Accessories Guide_ What You Need.docx
Wendy’s Menu Canada – Complete Guide 2025
How Food Data Scraping Is Revolutionizing Restaurant Growth Strategies
06. 1649335909782_CLI 250 NUTRITION_2022.pptx
08_Mango_Dis_PARTIALSTEMPARASITE.pdf -farmers
Lecture 2 Effect of water on shelf life of food.pptx
SUSTAINABLE FOOD PRODUCTION and supply chain
หลักสูตร Standard Barista for IPC barista
HealthyIndianBites:Eat Right, Live Right.pdf
1. CLEAN AND MAINTAIN KITCHEN PREMISES.pptx
Top 10 Viral Food Menus in 2025 | Menu Makanan Hits yang Lagi Viral!** **Must...
Food in the Maldives and Its Unique Culinary Tradition
ZDSPGC-PART-TIME-LOADING.docx and faculty
Brown-Illustrative-Abstract-Group-Project-Presentation-1.pdf
role of SGLT2s in Non-Diabetic Patients.pptx
FST-401 lecture # 12 food chemistry.pptx
Agricultural_Census_Main_Findings_Report.pdf
Aber毕业证学历认证,西蒙弗雷泽大学毕业证留学学历
Agrisphere ai powered presision farming marketplace
Food Chain and Food Web in the world.ppt

Genetic-Algorithms SDSDa SDD dfsAFF fsaf

  • 2. Introduction Introduction  After scientists became disillusioned with After scientists became disillusioned with classical and neo-classical attempts at classical and neo-classical attempts at modeling intelligence, they looked in other modeling intelligence, they looked in other directions. directions.  Two prominent fields arose, connectionism Two prominent fields arose, connectionism (neural networking, parallel processing) and (neural networking, parallel processing) and evolutionary computing. evolutionary computing.  It is the latter that this essay deals with - It is the latter that this essay deals with - genetic algorithms and genetic programming. genetic algorithms and genetic programming.
  • 3. What is GA What is GA  A A genetic algorithm genetic algorithm (or (or GA GA) is a search technique ) is a search technique used in computing to find true or approximate used in computing to find true or approximate solutions to optimization and search problems. solutions to optimization and search problems.  Genetic algorithms are categorized as global search Genetic algorithms are categorized as global search heuristics. heuristics.  Genetic algorithms are a particular class of Genetic algorithms are a particular class of evolutionary algorithms that use techniques inspired evolutionary algorithms that use techniques inspired by evolutionary biology such as inheritance, by evolutionary biology such as inheritance, mutation, selection, and crossover (also called mutation, selection, and crossover (also called recombination). recombination).
  • 4. What is GA What is GA  Genetic algorithms are implemented as a computer Genetic algorithms are implemented as a computer simulation in which a population of abstract simulation in which a population of abstract representations (called chromosomes or the genotype representations (called chromosomes or the genotype or the genome) of candidate solutions (called or the genome) of candidate solutions (called individuals, creatures, or phenotypes) to an individuals, creatures, or phenotypes) to an optimization problem evolves toward better solutions. optimization problem evolves toward better solutions.  Traditionally, solutions are represented in binary as Traditionally, solutions are represented in binary as strings of 0s and 1s, but other encodings are also strings of 0s and 1s, but other encodings are also possible. possible.
  • 5. What is GA What is GA  The evolution usually starts from a population of The evolution usually starts from a population of randomly generated individuals and happens in randomly generated individuals and happens in generations. generations.  In each generation, the fitness of every individual in In each generation, the fitness of every individual in the population is evaluated, multiple individuals are the population is evaluated, multiple individuals are selected from the current population (based on their selected from the current population (based on their fitness), and modified (recombined and possibly fitness), and modified (recombined and possibly mutated) to form a new population. mutated) to form a new population.
  • 6. What is GA What is GA  The new population is then used in the next The new population is then used in the next iteration of the algorithm. iteration of the algorithm.  Commonly, the algorithm terminates when Commonly, the algorithm terminates when either a maximum number of generations has either a maximum number of generations has been produced, or a satisfactory fitness level been produced, or a satisfactory fitness level has been reached for the population. has been reached for the population.  If the algorithm has terminated due to a If the algorithm has terminated due to a maximum number of generations, a maximum number of generations, a satisfactory solution may or may not have satisfactory solution may or may not have been reached. been reached.
  • 7. Key terms Key terms  Individual Individual - Any possible solution - Any possible solution  Population Population - Group of all - Group of all individuals individuals  Search Space Search Space - All possible solutions to the problem - All possible solutions to the problem  Chromosome Chromosome - Blueprint for an - Blueprint for an individual individual  Trait Trait - Possible aspect ( - Possible aspect (features) features) of an of an individual individual  Allele - - Possible settings of trait (black, blond, etc.)  Locus Locus - The position of a - The position of a gene gene on the on the chromosome chromosome  Genome Genome - Collection of all - Collection of all chromosomes chromosomes for an for an individual individual
  • 9. Genotype and Phenotype  Genotype: – Particular set of genes in a genome  Phenotype: – Physical characteristic of the genotype (smart, beautiful, healthy, etc.)
  • 11. GA Requirements GA Requirements  A typical genetic algorithm requires two things to be defined: A typical genetic algorithm requires two things to be defined:  a genetic representation of the solution domain, and a genetic representation of the solution domain, and  a fitness function to evaluate the solution domain. a fitness function to evaluate the solution domain.  A standard representation of the solution is as an array of bits. A standard representation of the solution is as an array of bits. Arrays of other types and structures can be used in essentially Arrays of other types and structures can be used in essentially the same way. the same way.  The main property that makes these genetic representations The main property that makes these genetic representations convenient is that their parts are easily aligned due to their convenient is that their parts are easily aligned due to their fixed size, that facilitates simple crossover operation. fixed size, that facilitates simple crossover operation.  Variable length representations may also be used, but Variable length representations may also be used, but crossover implementation is more complex in this case. crossover implementation is more complex in this case.  Tree-like representations are explored in Genetic Tree-like representations are explored in Genetic programming. programming.
  • 12. Representation Representation Chromosomes could be: Chromosomes could be:  Bit strings (0101 ... 1100) Bit strings (0101 ... 1100)  Real numbers (43.2 -33.1 ... 0.0 89.2) Real numbers (43.2 -33.1 ... 0.0 89.2)  Permutations of element (E11 E3 E7 ... E1 E15) Permutations of element (E11 E3 E7 ... E1 E15)  Lists of rules (R1 R2 R3 ... R22 R23) Lists of rules (R1 R2 R3 ... R22 R23)  Program elements (genetic programming) Program elements (genetic programming)  ... any data structure ... ... any data structure ...
  • 13. GA Requirements GA Requirements  The fitness function is defined over the genetic representation The fitness function is defined over the genetic representation and measures the and measures the quality quality of the represented solution. of the represented solution.  The fitness function is always problem dependent. The fitness function is always problem dependent.  For instance, in the For instance, in the knapsack problem we want to maximize we want to maximize the total value of objects that we can put in a knapsack of the total value of objects that we can put in a knapsack of some fixed capacity. some fixed capacity.  A representation of a solution might be an array of bits, where A representation of a solution might be an array of bits, where each bit represents a different object, and the value of the bit each bit represents a different object, and the value of the bit (0 or 1) represents whether or not the object is in the knapsack. (0 or 1) represents whether or not the object is in the knapsack.  Not every such representation is valid, as the size of objects Not every such representation is valid, as the size of objects may exceed the capacity of the knapsack. may exceed the capacity of the knapsack.  The The fitness fitness of the solution is the sum of values of all objects in of the solution is the sum of values of all objects in the knapsack if the representation is valid, or 0 otherwise. In the knapsack if the representation is valid, or 0 otherwise. In some problems, it is hard or even impossible to define the some problems, it is hard or even impossible to define the fitness expression; in these cases, interactive genetic fitness expression; in these cases, interactive genetic algorithms are used. algorithms are used.
  • 14. A fitness function A fitness function
  • 15. Basics of GA Basics of GA  The most common type of genetic algorithm works like this: The most common type of genetic algorithm works like this:  a population is created with a group of individuals created a population is created with a group of individuals created randomly. randomly.  The individuals in the population are then evaluated. The individuals in the population are then evaluated.  The evaluation function is provided by the programmer and The evaluation function is provided by the programmer and gives the individuals a score based on how well they perform gives the individuals a score based on how well they perform at the given task. at the given task.  Two individuals are then selected based on their fitness, the Two individuals are then selected based on their fitness, the higher the fitness, the higher the chance of being selected. higher the fitness, the higher the chance of being selected.  These individuals then "reproduce" to create one or more These individuals then "reproduce" to create one or more offspring, after which the offspring are mutated randomly. offspring, after which the offspring are mutated randomly.  This continues until a suitable solution has been found or a This continues until a suitable solution has been found or a certain number of generations have passed, depending on the certain number of generations have passed, depending on the needs of the programmer. needs of the programmer.
  • 16. General Algorithm for GA General Algorithm for GA  Initialization Initialization  Initially many individual solutions are randomly Initially many individual solutions are randomly generated to form an initial population. The generated to form an initial population. The population size depends on the nature of the problem, population size depends on the nature of the problem, but typically contains several hundreds or thousands but typically contains several hundreds or thousands of possible solutions. of possible solutions.  Traditionally, the population is generated randomly, Traditionally, the population is generated randomly, covering the entire range of possible solutions (the covering the entire range of possible solutions (the search space search space). ).  Occasionally, the solutions may be "seeded" in areas Occasionally, the solutions may be "seeded" in areas where optimal solutions are likely to be found. where optimal solutions are likely to be found.
  • 17. General Algorithm for GA General Algorithm for GA  Selection Selection  During each successive generation, a proportion of the During each successive generation, a proportion of the existing population is selected to breed a new generation. existing population is selected to breed a new generation.  Individual solutions are selected through a Individual solutions are selected through a fitness-based fitness-based process, where fitter solutions (as measured by a fitness process, where fitter solutions (as measured by a fitness function) are typically more likely to be selected. function) are typically more likely to be selected.  Certain selection methods rate the fitness of each solution and Certain selection methods rate the fitness of each solution and preferentially select the best solutions. Other methods rate preferentially select the best solutions. Other methods rate only a random sample of the population, as this process may only a random sample of the population, as this process may be very time-consuming. be very time-consuming.  Most functions are stochastic and designed so that a small Most functions are stochastic and designed so that a small proportion of less fit solutions are selected. This helps keep proportion of less fit solutions are selected. This helps keep the diversity of the population large, preventing premature the diversity of the population large, preventing premature convergence on poor solutions. Popular and well-studied convergence on poor solutions. Popular and well-studied selection methods include roulette wheel selection and selection methods include roulette wheel selection and tournament selection. tournament selection.
  • 18. General Algorithm for GA General Algorithm for GA  In roulette wheel selection, individuals are In roulette wheel selection, individuals are given a probability of being selected that is given a probability of being selected that is directly proportionate to their fitness. directly proportionate to their fitness.  Two individuals are then chosen randomly Two individuals are then chosen randomly based on these probabilities and produce based on these probabilities and produce offspring. offspring.
  • 19. General Algorithm for GA General Algorithm for GA Roulette Wheel’s Selection Pseudo Code: Roulette Wheel’s Selection Pseudo Code: for all members of population for all members of population sum += fitness of this individual sum += fitness of this individual end for end for for all members of population for all members of population probability = sum of probabilities + (fitness / sum) probability = sum of probabilities + (fitness / sum) sum of probabilities += probability sum of probabilities += probability end for end for loop until new population is full loop until new population is full do this twice do this twice number = Random between 0 and 1 number = Random between 0 and 1 for all members of population for all members of population if number > probability but less than next probability then if number > probability but less than next probability then you have been selected you have been selected end for end for end end create offspring create offspring end loop end loop
  • 20. General Algorithm for GA General Algorithm for GA  Reproduction Reproduction  The next step is to generate a second generation population of The next step is to generate a second generation population of solutions from those selected through genetic operators: solutions from those selected through genetic operators: crossover (also called recombination), and/or mutation. crossover (also called recombination), and/or mutation.  For each new solution to be produced, a pair of "parent" For each new solution to be produced, a pair of "parent" solutions is selected for breeding from the pool selected solutions is selected for breeding from the pool selected previously. previously.  By producing a "child" solution using the above methods of By producing a "child" solution using the above methods of crossover and mutation, a new solution is created which crossover and mutation, a new solution is created which typically shares many of the characteristics of its "parents". typically shares many of the characteristics of its "parents". New parents are selected for each child, and the process New parents are selected for each child, and the process continues until a new population of solutions of appropriate continues until a new population of solutions of appropriate size is generated. size is generated.
  • 21. General Algorithm for GA General Algorithm for GA  These processes ultimately result in the next These processes ultimately result in the next generation population of chromosomes that is generation population of chromosomes that is different from the initial generation. different from the initial generation.  Generally the average fitness will have Generally the average fitness will have increased by this procedure for the population, increased by this procedure for the population, since only the best organisms from the first since only the best organisms from the first generation are selected for breeding, along generation are selected for breeding, along with a small proportion of less fit solutions, for with a small proportion of less fit solutions, for reasons already mentioned above. reasons already mentioned above.
  • 22. Crossover Crossover  the most common type is single point crossover. In single the most common type is single point crossover. In single point crossover, you choose a locus at which you swap the point crossover, you choose a locus at which you swap the remaining alleles from on parent to the other. This is complex remaining alleles from on parent to the other. This is complex and is best understood visually. and is best understood visually.  As you can see, the children take one section of the As you can see, the children take one section of the chromosome from each parent. chromosome from each parent.  The point at which the chromosome is broken depends on the The point at which the chromosome is broken depends on the randomly selected crossover point. randomly selected crossover point.  This particular method is called single point crossover because This particular method is called single point crossover because only one crossover point exists. Sometimes only child 1 or only one crossover point exists. Sometimes only child 1 or child 2 is created, but oftentimes both offspring are created child 2 is created, but oftentimes both offspring are created and put into the new population. and put into the new population.  Crossover does not always occur, however. Sometimes, based Crossover does not always occur, however. Sometimes, based on a set probability, no crossover occurs and the parents are on a set probability, no crossover occurs and the parents are copied directly to the new population. The probability of copied directly to the new population. The probability of crossover occurring is usually 60% to 70%. crossover occurring is usually 60% to 70%.
  • 24. Mutation Mutation  After selection and crossover, you now have a new population After selection and crossover, you now have a new population full of individuals. full of individuals.  Some are directly copied, and others are produced by Some are directly copied, and others are produced by crossover. crossover.  In order to ensure that the individuals are not all exactly the In order to ensure that the individuals are not all exactly the same, you allow for a small chance of mutation. same, you allow for a small chance of mutation.  You loop through all the alleles of all the individuals, and if You loop through all the alleles of all the individuals, and if that allele is selected for mutation, you can either change it by that allele is selected for mutation, you can either change it by a small amount or replace it with a new value. The probability a small amount or replace it with a new value. The probability of mutation is usually between 1 and 2 tenths of a percent. of mutation is usually between 1 and 2 tenths of a percent.  Mutation is fairly simple. You just change the selected alleles Mutation is fairly simple. You just change the selected alleles based on what you feel is necessary and move on. Mutation is, based on what you feel is necessary and move on. Mutation is, however, vital to ensuring genetic diversity within the however, vital to ensuring genetic diversity within the population. population.
  • 26. General Algorithm for GA General Algorithm for GA  Termination Termination  This generational process is repeated until a This generational process is repeated until a termination condition has been reached. termination condition has been reached.  Common terminating conditions are: Common terminating conditions are:  A solution is found that satisfies minimum criteria A solution is found that satisfies minimum criteria  Fixed number of generations reached Fixed number of generations reached  Allocated budget (computation time/money) reached Allocated budget (computation time/money) reached  The highest ranking solution's fitness is reaching or has The highest ranking solution's fitness is reaching or has reached a plateau such that successive iterations no longer reached a plateau such that successive iterations no longer produce better results produce better results  Manual inspection Manual inspection  Any Combinations of the above Any Combinations of the above
  • 27. GA Pseudo-code GA Pseudo-code Choose initial population Choose initial population Evaluate the fitness of each individual in the population Evaluate the fitness of each individual in the population Repeat Repeat Select best-ranking individuals to reproduce Select best-ranking individuals to reproduce Breed new generation through crossover and mutation (genetic Breed new generation through crossover and mutation (genetic operations) and give birth to offspring operations) and give birth to offspring Evaluate the individual fitnesses of the offspring Evaluate the individual fitnesses of the offspring Replace worst ranked part of population with offspring Replace worst ranked part of population with offspring Until <terminating condition> Until <terminating condition>
  • 28. Symbolic AI VS. Genetic Symbolic AI VS. Genetic Algorithms Algorithms  Most symbolic AI systems are very static. Most symbolic AI systems are very static.  Most of them can usually only solve one given Most of them can usually only solve one given specific problem, since their architecture was specific problem, since their architecture was designed for whatever that specific problem was in designed for whatever that specific problem was in the first place. the first place.  Thus, if the given problem were somehow to be Thus, if the given problem were somehow to be changed, these systems could have a hard time changed, these systems could have a hard time adapting to them, since the algorithm that would adapting to them, since the algorithm that would originally arrive to the solution may be either originally arrive to the solution may be either incorrect or less efficient. incorrect or less efficient.  Genetic algorithms (or GA) were created to combat Genetic algorithms (or GA) were created to combat these problems; they are basically algorithms based these problems; they are basically algorithms based on natural biological evolution. on natural biological evolution.
  • 29. Symbolic AI VS. Genetic Symbolic AI VS. Genetic Algorithms Algorithms  The architecture of systems that implement genetic algorithms The architecture of systems that implement genetic algorithms (or GA) are more able to adapt to a wide range of problems. (or GA) are more able to adapt to a wide range of problems.  A GA functions by generating a large set of possible solutions A GA functions by generating a large set of possible solutions to a given problem. to a given problem.  It then evaluates each of those solutions, and decides on a It then evaluates each of those solutions, and decides on a "fitness level" (you may recall the phrase: "survival of the "fitness level" (you may recall the phrase: "survival of the fittest") for each solution set. fittest") for each solution set.  These solutions then breed new solutions. These solutions then breed new solutions.  The parent solutions that were more "fit" are more likely to The parent solutions that were more "fit" are more likely to reproduce, while those that were less "fit" are more unlikely to reproduce, while those that were less "fit" are more unlikely to do so. do so.  In essence, solutions are evolved over time. This way you In essence, solutions are evolved over time. This way you evolve your search space scope to a point where you can find evolve your search space scope to a point where you can find the solution. the solution.  Genetic algorithms can be incredibly efficient if programmed Genetic algorithms can be incredibly efficient if programmed correctly. correctly.
  • 30. Genetic Programming Genetic Programming  In programming languages such as LISP, the mathematical In programming languages such as LISP, the mathematical notation is not written in standard notation, but in prefix notation is not written in standard notation, but in prefix notation. Some examples of this: notation. Some examples of this:  + 2 1 + 2 1 : : 2 + 1 2 + 1  * + 2 1 2 * + 2 1 2 : : 2 * (2+1) 2 * (2+1)  * + - 2 1 4 9 : * + - 2 1 4 9 : 9 * ((2 - 1) + 4) 9 * ((2 - 1) + 4)  Notice the difference between the left-hand side to the right? Notice the difference between the left-hand side to the right? Apart from the order being different, no parenthesis! The Apart from the order being different, no parenthesis! The prefix method makes it a lot easier for programmers and prefix method makes it a lot easier for programmers and compilers alike, because order precedence is not an issue. compilers alike, because order precedence is not an issue.  You can build expression trees out of these strings that then You can build expression trees out of these strings that then can be easily evaluated, for example, here are the trees for the can be easily evaluated, for example, here are the trees for the above three expressions. above three expressions.
  • 32. Genetic Programming Genetic Programming  You can see how expression evaluation is thus a lot You can see how expression evaluation is thus a lot easier. easier.  What this have to do with GAs? If for example you What this have to do with GAs? If for example you have numerical data and 'answers', but no expression have numerical data and 'answers', but no expression to conjoin the data with the answers. to conjoin the data with the answers.  A genetic algorithm can be used to 'evolve' an A genetic algorithm can be used to 'evolve' an expression tree to create a very close fit to the data. expression tree to create a very close fit to the data.  By 'splicing' and 'grafting' the trees and evaluating By 'splicing' and 'grafting' the trees and evaluating the resulting expression with the data and testing it to the resulting expression with the data and testing it to the answers, the fitness function can return how close the answers, the fitness function can return how close the expression is. the expression is.
  • 33. Genetic Programming Genetic Programming  The limitations of genetic programming lie in The limitations of genetic programming lie in the the huge huge search space the GAs have to search search space the GAs have to search for - an infinite number of equations. for - an infinite number of equations.  Therefore, normally before running a GA to Therefore, normally before running a GA to search for an equation, the user tells the search for an equation, the user tells the program which operators and numerical ranges program which operators and numerical ranges to search under. to search under.  Uses of genetic programming can lie in stock Uses of genetic programming can lie in stock market prediction, advanced mathematics and market prediction, advanced mathematics and military applications . military applications .
  • 34. Evolving Neural Networks Evolving Neural Networks  Evolving the architecture of neural network is Evolving the architecture of neural network is slightly more complicated, and there have slightly more complicated, and there have been several ways of doing it. For small nets, a been several ways of doing it. For small nets, a simple matrix represents which neuron simple matrix represents which neuron connects which, and then this matrix is, in connects which, and then this matrix is, in turn, converted into the necessary 'genes', and turn, converted into the necessary 'genes', and various combinations of these are evolved. various combinations of these are evolved.
  • 35. Evolving Neural Networks Evolving Neural Networks  Many would think that a learning function could be Many would think that a learning function could be evolved via genetic programming. Unfortunately, evolved via genetic programming. Unfortunately, genetic programming combined with neural networks genetic programming combined with neural networks could be could be incredibly incredibly slow, thus impractical. slow, thus impractical.  As with many problems, you have to constrain what As with many problems, you have to constrain what you are attempting to create. you are attempting to create.  For example, in 1990, David Chalmers attempted to For example, in 1990, David Chalmers attempted to evolve a function as good as the delta rule. evolve a function as good as the delta rule.  He did this by creating a general equation based upon He did this by creating a general equation based upon the delta rule with 8 unknowns, which the genetic the delta rule with 8 unknowns, which the genetic algorithm then evolved. algorithm then evolved.
  • 36. Other Areas Other Areas  Genetic Algorithms can be applied to virtually any problem Genetic Algorithms can be applied to virtually any problem that has a large search space. that has a large search space.  Al Biles uses genetic algorithms to filter out 'good' and 'bad' Al Biles uses genetic algorithms to filter out 'good' and 'bad' riffs for jazz improvisation. riffs for jazz improvisation.  The military uses GAs to evolve equations to differentiate The military uses GAs to evolve equations to differentiate between different radar returns. between different radar returns.  Stock companies use GA-powered programs to predict the Stock companies use GA-powered programs to predict the stock market. stock market.
  • 37. Example Example  f(x) = {MAX(x f(x) = {MAX(x2 2 ): 0 <= x <= 32 } ): 0 <= x <= 32 }  Encode Solution: Just use 5 bits (1 or 0). Encode Solution: Just use 5 bits (1 or 0).  Generate initial population. Generate initial population.  Evaluate each solution against objective. Evaluate each solution against objective. Sol. Sol. String String Fitness Fitness % of Total % of Total A A 01101 01101 169 169 14.4 14.4 B B 11000 11000 576 576 49.2 49.2 C C 01000 01000 64 64 5.5 5.5 D D 10011 10011 361 361 30.9 30.9 A A 0 0 1 1 1 1 0 0 1 1 B B 1 1 1 1 0 0 0 0 0 0 C C 0 0 1 1 0 0 0 0 0 0 D D 1 1 0 0 0 0 1 1 1 1
  • 38. Example Cont’d Example Cont’d  Create next generation of solutions Create next generation of solutions  Probability of “being a parent” depends on the fitness. Probability of “being a parent” depends on the fitness.  Ways for parents to create next generation Ways for parents to create next generation  Reproduction Reproduction  Use a string again unmodified. Use a string again unmodified.  Crossover Crossover  Cut and paste portions of one string to another. Cut and paste portions of one string to another.  Mutation Mutation  Randomly flip a bit. Randomly flip a bit.  COMBINATION of all of the above. COMBINATION of all of the above.
  • 39. Checkboard example  We are given an n by n checkboard in which every field can have a different colour from a set of four colors.  Goal is to achieve a checkboard in a way that there are no neighbours with the same color (not diagonal) 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
  • 40. Checkboard example Cont’d  Chromosomes represent the way the checkboard is colored.  Chromosomes are not represented by bitstrings but by bitmatrices  The bits in the bitmatrix can have one of the four values 0, 1, 2 or 3, depending on the color.  Crossing-over involves matrix manipulation instead of point wise operating.  Crossing-over can be combining the parential matrices in a horizontal, vertical, triangular or square way.  Mutation remains bitwise changing bits in either one of the other numbers.
  • 41. Checkboard example Cont’d • This problem can be seen as a graph with n nodes and (n-1) edges, so the fitness f(x) is defined as: f(x) = 2 · (n-1) ·n
  • 42. Checkboard example Cont’d • Fitnesscurves for different cross-over rules: 0 100 200 300 400 500 130 140 150 160 170 180 Fitness Lower-Triangular Crossing Over 0 200 400 600 800 130 140 150 160 170 180 Square Crossing Over 0 200 400 600 800 130 140 150 160 170 180 Generations Fitness Horizontal Cutting Crossing Over 0 500 1000 1500 130 140 150 160 170 180 Generations Verical Cutting Crossing Over