A Comparison of Particle Swarm Optimization and Differential Evolution

DOI: 10.5121/ijsc.2012.3302 13
A COMPARISON OF PARTICLE SWARM
OPTIMIZATION AND DIFFERENTIAL EVOLUTION
Vu Truong Vu
Ho Chi Minh City University of Transport, Faculty of Civil Engineering
No.2, D3 Street, Ward 25, Binh Thanh District, HCMC, Viet Nam
vutruongvu@gmail.com
ABSTRACT
Two modern optimization methods including Particle Swarm Optimization and Differential Evolution are
compared on twelve constrained nonlinear test functions. Generally, the results show that Differential
Evolution is better than Particle Swarm Optimization in terms of high-quality solutions, running time and
robustness.
KEYWORDS
Particle Swarm Optimization, Differential Evolution
1. INTRODUCTION
As early as 1975, Holland [1] published the primary book of Genetic Algorithms, one of the most
attractive modern optimization methods so far. Other modern methods have been introduced
since then. Some well-known methods are listed chronologically here. Kirkpatrick et al. [2] with
Simulated Annealing in 1983, Glover [3] with Tabu Search in 1986, Dorigo et al. [4] with Ant
System in 1991 and later version Ant Colony Optimization [5] in 1999, Kennedy and Eberhart
[6, 7] with Particle Swarm Optimization in 1995, Storn and Price [8] with Differential Evolution
in 1995. It should be noted that all modern optimization techniques in this study belong to
stochastic class of optimization and have some common characteristics such as being based on
nature, using probabilistic rules, and only using objective function information. The methods
have proven their usefulness in real problems in which the objective functions and constraints are
not continuous and non-differentiable.
The “No Free Lunch” theorem [9] states that no algorithm is perfect or in another words, each
method has its own advantages and disadvantages. As a result, each method is only effective in
certain problems. However this does not mean that all comparisons are unnecessary. At most, a
comparison provides insight into the algorithms. At least, it shows which type of problem is
suitable for which algorithm. From this light, one can select the most suitable each algorithm for a
particular situation. Two methods Swarm Optimization and Differential Evolution which are
originally applied to continuous variables will be compared in this study.
2. PARTICLE SWARM OPTIMIZATION
Particle Swarm Optimization (PSO), proposed by Kennedy and Eberhart in 1995 [6, 7, 10],
simulates the behaviour of bird flocks searching for targets. During searching, each individual

International Journal on Soft Computing (IJSC) Vol.3, No.3, August 2012
14
continuously adjusts its position/movement according to its own experience (its best position so
far) and its neighbourhood experience (the best position obtained by its neighbourhood). This
simulation was introduced to optimization of nonlinear, multi-dimensional functions with the
similarity given in Table 1.
In the original version of PSO [6, 7], each particle position and velocity are updated in the form:
)
x
-
p
(
cr
+
)
x
-
p
(
cr
+
v
=
v k
ij
k
gj
k
j
2
k
ij
k
ij
k
j
1
k
ij
1
+
k
ij (1)
1
+
k
ij
k
ij
1
+
k
ij v
+
x
=
x (2)
where i = 1, 2, . . ., N, and N is the size of the swarm
j = 1, 2, . . ., D, and D is the number of dimensions of the problem
k = 1, 2, . . ., Iter, and Iter is the maximum iteration number
k
ij
x is the position of particle i, dimension j, at iteration k. The position is also parameter
of the problem
k
ij
v is the velocity (position change in a unit time step) of particle i, dimension j, at
iteration k
c is an acceleration constant, c = 2
r1, r2 are independent random numbers, uniformly distributed in (0, 1)
k
ij
p is the personal best position of particle i, dimension j, at iteration k
k
gj
p is the neighbourhood’s best position, dimension j, at iteration k
There are two models of PSO, one with a global neighbourhood and another with a local
neighbourhood. In the global model, a particle has information of its own and of the entire
population, whereas in the local model, a particle has information of its own and its nearest
neighbours. The global model, which is faster than the local model [11], is described in the
algorithm for a minimization problem as shown in Fig. 1. Based on the basic Eq. (1), many
variants of PSO are developed. Each part in the right-hand side of Eq. (1) has its own meaning.
- The first term, vij, is a momentum. An attempt to remove this value from the algorithm leads to
ineffectiveness at finding global optima, as Kennedy et al. [7] pointed out.
- The second term is the cognitive component that relates to individual experience. This
component represents the distance from current particle position to its best position so far. The
cognitive component represents the natural tendency of individuals to return to environments
where they experienced their best performance [12].
- The third term is the social component that shows the collaboration among particles. It
represents the distance from a current particle position to the best position of the whole swarm (in
global model) or its neighbourhood (in local model). This component represents the tendency of
individuals to follow the success of other individuals [12].
Two versions of PSO will be studied in the paper. They are Inertia weight variant introduced by
Shi et al. [13, 14], and Constriction factor variant proposed by Clerc [15], Clerc et al. [16].

15
Table 1 The similarity between bird flock’s properties and optimization problem in
Particle Swarm Optimization
Bird flock’s properties Optimization problem
Particle (or Individual) Variable
Swarm (or Population) Set of variables
Neighbourhood Subset/Set of variables
3 dimensions D dimensions
Particle position Variable value
Particle velocity Variable increment
Search target Objective function
2.1. Inertia weight variant (PSO-1)
Shi et al. [13] introduced a modified PSO in which the Eq. (1) is changed into
)
x
(p
r
c
)
x
(p
r
c
ωv
v k
ij
k
gj
k
2j
2
k
ij
k
ij
k
1j
1
k
ij
1
k
ij −
+
−
+
=
+
(3)
where, ω is the inertia weight; c1, c2 are two positive constants, called cognitive and social
parameters respectively.
The velocity in Eq. (3) is limited by a maximum velocity, vmax. This limit reduces the excessive
step size in Eq. (2). The maximum velocity is computed as
vmax = cv(xmax – xmin) (4)
where cv is a maximum velocity coefficient, xmax and xmin are the maximum and minimum
positions of a particle.
The recommended choice in [7] for constants c1 and c2 is 2 since on average it makes the weights
for cognitive and social parts to be 1.
Coefficient ω is used to contain the explosion as particles velocities and positions move toward a
boundary. While a large inertia weight ω favours a global search, a small value supports a local
search. A suitable selection of the inertia weight can provide a balance between global and local
exploration abilities [14]. Playing such a role, naturally its value should be large in early phase of
search in order to reduce omitting potential targets. Once a search area is found, small value of ω
may be suitable for refining the search. As proposed in [14], ω decreases linearly from 0.9 to 0.4
combined with the limit of the velocity vmax.
2.2. Constriction factor variant (PSO-2)
Instead of using the inertia weight and velocity clamping, Clerc [15], Clerc et al. [16] introduced
a constriction factor χ. In this case, the Eq. (1) becomes

16
))
x
(p
r
c
)
x
(p
r
c
χ(v
v k
ij
k
gj
k
2j
2
k
ij
k
ij
k
1j
1
k
ij
1
k
ij −
+
−
+
=
+
(5)
where
|
4
2
|
2
χ
2
ϕ
ϕ
ϕ −
−
−
= , φ = c1 + c2, and φ > 4
This method can prevent explosion and more importantly induces particles to converge on a
steady state. Unlike the inertia weight variant, this method does not need a velocity limit [16],
although the use of it may improve performance, for example, in [17] and [18]. Comparing
inertia weights and constriction factors can be seen in [17] where the conclusion is in favour of
the constriction factor variant.
2.3. Suggestions on choice of parameters
Some authors investigated the optimal choice of parameters. Eberhart et al. [19] showed that
population size N is problem-dependent. Sizes of 20-50 are most common. This size is usually
smaller than that of other evolutionary algorithms. Carlisle et al. [18] showed that for general
problems, the optimal value of φ is 4.1 along with c1 = 2.8, c2 = 1.3, population N = 30 and
maximum velocity vmax is set to zero at maximum position xmax.
For each particle xi, i = 1, N
Initialize random positions and velocities on D dimensions.
Evaluate objective function, f(xi)
Assign pi = xi and f(pi) = f(xi)
If f(xi) < f(pg) then
f(pg) = f(xi)
pg = xi
End If
End For
Repeat
For each particle xi, i = 1, N
Update particle velocity and position using
)
-
(
)
-
( 2
1
1 k
ij
k
gj
k
j
k
ij
k
ij
k
j
k
ij
k
ij x
p
cr
x
p
cr
v
v +
+
=
+
, j = 1, D
1
+
k
ij
k
ij
1
+
k
ij v
+
x
=
x
Evaluate objective function, f(xi)
Compare f(xi) with particle’s previous best value, f(pi)
If f(xi) < f(pi) then
f(pi) = f(xi)
pi = xi
End If
Compare f(xi) with neighbourhood’s previous best, f(pg)
If f(xi) < f(pg) then
f(pg) = f(xi)
pg = xi
End If
End For
Until maximum iteration or minimum criterion is met.
Fig. 1 Particle Swarm Optimization algorithm for minimization problem.

17
3. DIFFERENTIAL EVOLUTION
Differential Evolution (DE), invented by Storn and Price [8, 20-22], is based on the idea of
employing information within the vector population to alter the search space. This operation is
called “mutation”. DE also borrows the idea from Evolution Strategy such as “population”,
“crossover” in which components are mixed, and “selection” in which the best is reserved for the
next generation.
Let us consider a population of D-dimensional vectors xi,G i = 1, 2,…, N as a population for each
generation G, vi,G+1 as a mutant vector in generation (G + 1) and ui,G+1 as a trial vector in
generation (G + 1). Three operators in DE are described as follows.
• Mutation:
vi,G+1 = xr1,G + FM(xr2,G – xr3,G) i = 1, 2,…, N (6)
where r1, r2, r3 are random numbers in [1, N], integer, mutually different, and different from the
running index i; FM is mutation constant in [0, 2] ], i.e. 0 ≤ FM ≤ 2.
• Crossover:
ui,G+1 = (u1i,G+1, u2i,G+1,…, uDi,G+1) (7)
where 2,...D
1,
j
,
k
j
and
CR)
(r
if
x
k
j
or
CR)
(r
if
v
u
G
ji,
1
G
ji,
1
G
ji, =



≠
>
=
≤
=
+
+
CR is the crossover constant in [0, 1], r is random number in (0, 1) ), i.e. 0 < r < 1, k is a random
integer number in [1, D] which ensures that ui,G+1 gets at least one component from vi,G+1.
• Selection:
otherwise
x
)
f(x
)
f(u
if
u
x
G
i,
G
i,
1
G
i,
1
G
i,
1
G
i,


 <
=
+
+
+ (8)
where f(xi,G+1) is the objective function.
For this variant of DE, the notation DE/rand/1/bin (DE-1, for brevity) is introduced where rand
denotes a randomly chosen population vector for mutation, 1 is the number of difference vectors
used, and bin is the crossover scheme due to independent binomial experiments. Algorithm of this
variant for a minimization problem is shown in Fig. 2.
Another variant proved to be useful is DE/best/2/bin (DE-2, for brevity), where best means the
vector of lowest objective function to be mutated, 2 is the number of difference vectors used [22].
In this variant, Eq. (6) becomes
vi,G+1 = xbest,G + FM(xr1,G + xr2,G – xr3,G – xr4,G) (9)
According to [23], variant DE-2 seems to be better than DE-1 because DE-2 can generate more
different trial vectors than DE-1. For example, with a population of six individuals, 360 different

18
trial vectors can be generated by DE-2 but only 30 for DE-1. On the other hand, using the best
vector in DE-2 makes the population tend to the local minimum.
3.1. Suggestion on choice of parameters
There are three parameters in DE with the range suggested as follows.
• Population size N: N is at least 4 in DE-1 or at least 5 in DE-2 to make sure there are
sufficient materials for mutation. It ranges between 5*D and 10*D in [20], or
between 1.5*D and 2*D in [24], or from 3*D to 8*D in [23], from 1.5*D to 3.8*D in
optimization of four nonlinear constrained functions [25].
• Mutation constant FM: It is normally between 0.4 and 1, with 0.5 as a good initial
choice. If premature convergence occurs, N and/or FM should be increased [20].
Larger FM increases the probability of escaping a local minimum but for FM > 1 the
convergence decreases. A good initial choice is 0.6 [23]. Choice of 0.9 in
optimization of four nonlinear constrained functions [25] is reasonable.
• Crossover constant CR: The larger the value of CR, the faster and riskier the
convergence. A first good choice is 0.9 or 1 to quickly see the possible solution [20],
a choice of 0.1 will slow down convergence but be more thorough. According to
[23], CR varies form 0.3 to 0.9. A suitable value of 0.9 in optimization of four
nonlinear constrained functions is given in [25].
4. CONSTRAINT HANDLING TECHNIQUES
The two optimization methods mentioned above were originally developed for unconstrained
optimization problems. In order to apply them to constrained optimization problems, a typical
form of most real-world problems, constraint-handling techniques are needed. Comprehensive
overviews of the most popular constraint-handling techniques currently used with heuristic
methods can be found in Michalewicz and Fogel [26], Coello [27]. In the paper, a method that
makes a clear distinction between feasible solutions and infeasible solutions will be adopted.
In this method, the comparison of two solutions follows the rule suggested by Jimenez et al. [28]:
(i) for two feasible solutions, the one with better objective function value is chosen, (ii) for one
feasible solution and one infeasible solution, the feasible solution is chosen, and (iii) for two
infeasible solutions, the one with smaller constraint violation is chosen. This method requires no
additional coefficient and gives a natural approach to the feasible zone from trials in the
infeasible zone. Pulido et al. [29], Zavala et al. [30] applied the similar rule to PSO. Lampinen
[25] and Montes et al. [31] also applied this rule to DE.

19
For i = 1, N
For j = 1, D
Initialize random values xji,G1
End For
Evaluate objective function, f(xi,G1)
End For
Repeat
For i = 1, N
Generate 3 integer random numbers r1 ≠ r2 ≠ r3 ≠ i∈ [1,
N]
Generate integer random number k∈[1, D]
For j = 1, D
Generate random number r∈(0,1)
If ((r ≤ CR) or (j = D)) then
uki,G2 = xjr1,G1 + FM(xjr2,G1 – xjr3,G1)
Else
uki,G2 = xji,G1
End If
k = (k + 1) modulo D
End For
Evaluate objective function, f(ui,G2)
If f(ui,G2) < f(xi,G1) then
xji,G2 = uji,G1
Else
xji,G2 = xji,G1
End If
End For
For i = 1, N
For j = 1, D
Update xji,G1 = xji,G2
End For
End For
Until maximum iteration or minimum criterion is met.
Fig. 2 Differential Evolution algorithm (DE-1) for minimization problem.
5. SETTING NUMERICAL EXPERIMENTS
5.1. The experimentation goals and performance measures
The experimentation goals should be considered when assessing the effectiveness of algorithms
and comparing them. Barr [32] suggested the following criteria:
- Accurate: identifying higher-quality solutions than other approaches.
- Fast: producing high-quality solutions quicker than other approaches.
- Robust: less sensitive to differences in problem characteristics and choices of the
initial variables than other approaches.
- Simple: easy to implement (less number of code lines and parameters).

20
5.2. Test functions
A constrained optimization problem has the following type:
Minimise f(x) x∈Rn
(10)
Subject to gj(x) ≥ 0, j = 1, …, q
hk(x) = 0, k = 1, …, p
u
i
i
l
i x
x
x ≤
≤ , i = 1, …, n
where x is a vector of size n, f(x) is the objective function, gj(x) is the jth
inequality constraint,
hk(x) is the kth
equality constraints, and ]
x
,
[x u
i
l
i are the lower and upper bounds of the variable xi.
Twelve test functions have been taken randomly from Hock and Schittkowski [33], Himmelblau
[34]. Information on them is shown in Table 2 and details of functions can be seen in the
appendix. Test functions are grouped in four objective function types: linear, quadratic,
polynomial and general type. Each type in turn has three constraint types: quadratic, polynomial
and general type. The published optimal or best-known solutions, xmin, of all test examples were
found with constraint violations which are below a tolerance, ε, of 10-7
. For compatibility with
published solutions, the tolerance will be the same, i.e., ε =10-7
.
Table 2 Information of test functions
Function Objective
function
type
Constraint
type
Inequality
constraints
Equality
constraints
No. of
variables
Best-known
solution, fmin
f1 Linear Quadratic 6 0 8 7049.330923
f2 Linear Polynomial 2 0 4 727.67937
f3 Linear General 2 0 3 0.5181632741
f4 Quadratic Quadratic 6 0 5 -30665.53867
f5 Quadratic Polynomial 1 0 2 1.0
f6 Quadratic General 8 3 10 -1768.80696
f7 Polynomial Quadratic 2 0 2 -6961.81381
f8 Polynomial Polynomial 1 1 4 17.0140173
f9 Polynomial General 2 3 4 5126.4981
f10 General Quadratic 3 0 2 -7.804226324
f11 General Polynomial 0 3 5 0.0539498478
f12 General General 0 3 10 -47.76109026
5.3. Setting parameters of methods
Parameters are based on suggestions of each method as mentioned above. Combinations of
parameters are also tried to find the best one. Each function will be tested with 50 independent
runs. The number of trials for each run is fixed, Imax = 3x106
. Parameters for variants of PSO and
DE are used as follows.
Population, N 30
Maximum iteration, Iter 105
( No. of maximum trial Imax = N x Iter = 3x106
)
For variant PSO-1 c1 = 2, c2 = 2, ω decreases linearly from 0.9 to 0.4, cv = 0.5
For variant PSO-2 There are 9 pairs of (c1, c2) which satisfy the relation c1+c2 = 4.1
to be used for determination of the best combination. They are
(c1, c2) = (0.3, 3.8), (1.0, 3.1), (1.3, 2.8), (1.7, 2.4), (2.05, 2.05),
(2.4, 1.7), (2.8, 1.3), (3.1, 1.0), (3.8, 0.3).

21
For variants DE-1, DE-2 There are 6 pairs of (CR, FM) to be used for determination of the
best combination, including (CR, FM) = (0.1, 0.5), (0.1, 1.0),
(0.5, 0.5), (0.5, 1.0), (1.0, 0.5), (1.0, 1.0)
6. RESULTS
6.1. Quality of solutions
• As shown in Table 3, PSO-1 obtained 2 solutions better than best-known solutions while
the corresponding values of PSO-2, DE-1, DE-2 are 3, 5 and 7 respectively. Generally,
the solution quality decreases in the order: DE-2, DE-1, PSO-2 and PSO-1. Therefore,
two variants DE-2 and PSO-2 represented for the two methods will be discussed next.
• For PSO-2, combinations with c1 ≥ c2 give better solutions than those with c1 c2. For
DE-2, in overall the combination (CR = 1.0, FM = 0.5) is the best.
• For all test functions, the best solutions of DE are always better than or as good as those
of PSO. In addition, other best results of DE are approximate to best-known solutions,
except the function f6. Reason of discrepancy is still unknown and this makes the use of
DE and PSO as global optimization methods with some carefulness. Best solutions of
methods are listed in Table 4. Details of result are given in Table 5 and Table 6.
6.2. Running time
Besides the quality of solution which is the most important factor for evaluation of an algorithm,
running time only has meaning if the solution quality is good enough. In this light, the results in
Table 7 show that running time of DE-2 is faster than that of PSO-2 from 1.1 to 4.4 times
depending on problems, except for function f10.
6.3. Robustness
• From Table 7, it is seen that the number of runs which found best solutions of DE-2 are
always higher than or equal to that of PSO-2. The rate of feasible trials is the ratio
between the number of feasible trials and the total trials. In general, the rate of feasible
trials of DE-2 is higher than that of PSO-2. Besides, number of runs in DE-2 which could
not find any feasible solution is the least.
• Again, from Table 5 and Table 6, combinations with c1 ≥ c2 give better rate of feasible
trials than combinations with c1 c2 in PSO-2; and the combination (CR = 1.0, FM = 0.5)
is the best for the rate of feasible trials in DE-2.
6.4. Simplicity
DE-2 and PSO-2 have the same 4 parameters. For method complexity, there are about 40 code
lines for PSO-2 and 70 code lines for DE-2 in the main loop.
7. CONCLUSION
Generally, DE-2 is better than PSO-2 in terms of solution quality, running time and chance of
reaching the best solutions in a variety of problems. In the other hand, number of code lines of
PSO-2 is the least. In the author’s viewpoint, the best method for the twelve test functions is DE-
2. It is also worth noting that this conclusion is based on results of test functions which cover a

22
variety of types such as linear, quadratic, polynomial and general functions; with given
parameters and the constraint handling techniques associated with the algorithms.
Table 3 Minimum values of objective function after 50 testing runs
(Bold values mean that the solutions are equal to or better than best-known solutions)
Func.
Minimum value of objective functions
PSO-1 PSO-2 DE-1 DE-2 Best-known
f1 7469.04471 7053.817430 7049.246945 7049.246506 7049.330923
f2 727.674508 727.674485 727.674443 727.674443 727.679370
f3 0.5181667914 0.5436562994 0.5181631954 0.5181631954 0.5181632741
f4 -30665.53902 -30665.53902 -30665.53902 -30665.53902 -30665.53867
f5 0.9907383666 0.9907383666 0.9907383666 0.9907383666 1.0000
f6 -1221.54117 -1477.62925 -1278.07878 -1690.00300 -1768.80696
f7 -6961.81111 -6961.81111 -6961.81111 -6961.81111 -6961.81381
f8 23.8848784 19.0024993 20.2730530 17.0140172 17.0140173
f9 5126.6299 5126.9172 5126.5157 5126.4983 5126.4981
f10 -7.802786736 -7.802786736 -7.802786736 -7.802786736 -7.804226324
f11 0.4561284690 0.0645899015 0.4524708138 0.0539498394 0.0539498478
f12 -42.67380277 -47.37065154 -47.67061450 -47.44572377 -47.76109026
Table 4 Best solutions, xmin, of two methods
Function PSO-2 DE-2
f1
(634.3674968, 1250.0134128,
5169.4365273, 186.4438325,
293.2225136, 213.5562164,
293.2213678, 393.2225258)
(579.4606149, 1359.8756712,
5109.9102202, 182.0305765,
295.6035661, 217.9694724,
286.4270592, 395.6035784)
f2
(193.4090071, 179.5210305,
185.0854729, 168.6589740)
(193.4077513, 179.5462924,
185.0160256, 168.7043735)
f3 (0.0, 0.9999998, 2.7182814) (0.1841265, 1.2021678, 3.3273220)
f4
(78.0, 33.0, 29.9952492,
45.0, 36.7758311)
(78.0, 33.0, 29.9952492,
45.0, 36.7758311)
f5 (1.0046415, 0.0) (1.0046415, 0.0)
f6
(1746.9672310, 15995.7276777,
63.2719138, 3071.2845436,
2000.0, 87.3578034,
94.0349684, 10.3011250,
2.5096923, 150.6019638)
(1648.4425610, 14406.1129388,
57.2832896, 2935.8537982,
1933.2991566, 90.7273299,
95.0, 9.9120298,
1.5640451, 153.5349454)
f7 (14.0950011, 0.8429632) (14.0950011, 0.8429632)
f8 (1.0, 3.4347215, 5.0, 1.4841454) (1.0, 4.7429996, 3.8211499, 1.3794082)
f9
(688.8741362, 1016.5435397,
0.1125169, -0.3992564)
(679.9453373, 1026.0671532,
0.1188763, -0.3962335)
f10 (13.5501432, 51.6599703) (13.5501432, 51.6599705)
f11
(-1.7130059, 1.5909154, 1.7775005,
0.5445980, 1.0385103)
(-1.7171435, 1.5957097, 1.8272457,
0.7636430, 0.7636430)
f12
(-96.6741714, -98.6387642,
-1.5556892, -6.5901656,
-0.6945218, -100.0,
-99.5173022, -99.9970261,
-41.9674309, -15.1797737)
(-3.2354681, -8.0191455,
-0.0202026, -82.8016103,
-0.6931470, -41.2589767,
-92.5399330, -4.7868082,
-5.1446518, -17.5464207)

23
Table 5 Results of PSO-2 with different combinations (c1, c2)

24
Table 6 Results of DE-2 with different combinations (CR, FM)

25
Table 7 Data after 50 testing runs, each test has 3x106
trials
(Bold values mean that the solutions are equal to or better than best-known solutions)
Func.
Running time per test
(s)
Number of
runs which
found best
solutions
Rate of
feasible trials
(%)
Number of
runs which could
not find any feasible
solution
PSO-2
(t1)
DE-2
(t2)
t1/t2 PSO-2
DE-
2
PSO-2 DE-2 PSO-2 DE-2
f1 21.439 4.924 4.4 1 1 67.115 36.033 0 0
f2 5.292 4.111 1.3 1 48 85.047 46.150 0 0
f3 14.107 4.392 3.2 47 12 75.954 73.155 3 0
f4 17.461 7.453 2.3 50 50 77.725 99.307 0 0
f5 15.333 5.647 2.7 2 13 99.997 99.995 0 0
f6 16.312 7.945 2.1 1 1 6.489 32.712 11 11
f7 12.673 8.872 1.4 43 45 67.338 77.638 5 5
f8 8.356 7.289 1.1 1 4 4.612 81.748 0 0
f9 15.419 7.067 2.2 1 4 49.283 65.354 11 0
f10 15.587 22.051 0.7 1 1 94.983 99.428 0 0
f11 12.456 11.483 1.1 1 1 15.794 45.375 0 0
f12 36.630 18.408 2.0 1 1 43.428 49.358 0 0
REFERENCES
[1] Holland, J. H., (1975), Adaptation in natural and artificial systems, University of Michigan Press Ann
Arbor.
[2] Kirkpatrick, S., Gelatt, C. D., and Vecchi, M. P., (1983), Optimization by Simulated Annealing,
Science, Vol. 220, No. 4598, pp. 671-680.
[3] Glover, F., (1986), Future paths for integer programming and links to artificial intelligence,
Computers and Operations Research, Vol. 13, No. 5, pp. 533-549.
[4] Dorigo, M., Maniezzo, V., and Colorni, A., (1991), Positive feedback as a search strategy,
Technical Report 91-016, Dipartimento di Elettronica, Politecnico de Milano, Italy.
[5] Dorigo, M., and Caro, G. D., (1999), The ant colony optimization meta-heuristic, New Ideas in
Optimization, D. Corne, M. Dorigo, and F. Glover, eds., McGraw-Hill, Maidenhead, UK.
[6] Eberhart, R., and Kennedy, J., (1995), A new optimizer using particle swarm theory, Proc.
Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya,
Japan, Piscataway, NJ: IEEE Service Center, pp. 39-43.
[7] Kennedy, J., and Eberhart, R., (1995), Particle swarm optimization, Proc. Proceedings of IEEE
International Conference on Neural Networks, Piscataway, NJ, pp. 1942-1948.
[8] Storn, R., and Price, K., (1995), Differential Evolution - A simple and efficient adaptive scheme for
global optimization over continuous spaces, Technical Report TR-95-012, International Computer
Science Institute, Berkeley.
[9] Wolpert, D. H., and Macready, W. G., (1997), No free lunch theorems for optimization, IEEE
Transactions on Evolutionary Computation, Vol. 1, No. 1, pp. 67-82.
[10] Kennedy, J., Eberhart, R. C., and Shi., Y., (2001), Swarm intelligence, Morgan Kaufmann Publishers,
San Francisco.
[11] Hu, X., and Russell, C. E., (2002), Solving constrained nonlinear optimization problems with
particle swarm optimization, Proceedings of the 6th World Multiconference on Systemics,
Cybernetics and Informatics (SCI 2002) Orlando, USA.
[12] Van Den Bergh, F., and Engelbrecht, A. P., (2006), A study of particle swarm optimization particle
trajectories, Information Sciences, Vol. 176, No. 8, pp. 937-971.

26
[13] Shi, Y., and Eberhart, R., (1998), A modified particle swarm optimizer, Proc. Proceedings of the
IEEE Conference on Evolutionary Computation, pp. 69-73.
[14] Shi, Y., and Eberhart, R. C., (1998), Parameter selection in particle swarm optimization, Proc.
Evolutionary Programming VII: Proceedings of the Seventh Annual Conference on Evolutionary
Programming,, V. W. Porto, N. Saravanan, D. E. Waagen, and A. E. Eiben, eds., Springer-Verlag,
NewYork, pp. 591-600.
[15] Clerc, M., (1999), The swarm and the queen: Towards a deterministic and adaptive particle swarm
optimization, Proc. Proceedings of the 1999 Congress on Evolutionary Computation, P. J. Angeline,
Z. Michalewicz, M. Schoenauer, X. Yao, and A. Zalzala, eds., pp. 1951-1957.
[16] Clerc, M., and Kennedy, J., (2002), The particle swarm-explosion, stability, and convergence in a
multidimensional complex space, IEEE Transactions on Evolutionary Computation, Vol. 6, No. 1,
pp. 58-73.
[17] Eberhart, R. C., and Shi, Y., (2000), Comparing inertia weights and constriction factors in particle
swarm optimization, Proc. Proceedings of the IEEE Conference on Evolutionary Computation, San
Diego, CA., pp. 84-88.
[18] Carlisle, A., and Dozier, G., (2001), An off-the-shelf PSO, Proc. Proceedings of the Workshop on
Particle Swarm Optimization, Indianapolis, IN, pp. 1-6.
[19] Eberhart, R. C., and Shi, Y., (2001), Particle swarm optimization: Developments, applications and
resources, Proc. Proceedings of the IEEE Conference on Evolutionary Computation, ICEC, pp. 81-
86.
[20] Storn, R., and Price, K., (1997), Differential Evolution - A Simple and Efficient Heuristic for Global
Optimization over Continuous Spaces, Journal of Global Optimization, Vol. 11, No. 4, pp. 341-359.
[21] Storn, R., and Price, K., (1996), Minimizing the real functions of the ICEC'96 contest by differential
evolution, Proc. Proceedings of the IEEE Conference on Evolutionary Computation, Nagoya, Japan,
pp. 842-844.
[22] Price, K. V., (1996), Differential evolution: a fast and simple numerical optimizer, Proc. Biennial
Conference of the North American Fuzzy Information Processing Society - NAFIPS,, M. Smith, M.
Lee, J. Keller, and J. Yen, eds., IEEE Press, New York, pp. 524-527.
[23] Gamperle, R., Muller, S. D., and Koumoutsakos, P., (2002), A parameter study for differential
evolution, Proc. Advances in Intelligent Systems, Fuzzy Ststems, Evolutionary Computation,, A.
Grmela, and N. E. Mastorakis, eds., WSEAS Press, pp. 293-298.
[24] Mayer, D. G., Kinghorn, B. P., and Archer, A. A., (2005), Differential evolution - An easy and
efficient evolutionary algorithm for model optimisation, Agricultural Systems, Vol. 83, No. 3, pp.
315-328.
[25] Lampinen, J., (2001), Solving problems subject to multiple nonlinear constraints by the differential
evolution, Proc. Proceedings of MENDEL 2001, 7th International Conference on Soft Computing,
R. Matousek, and P. Osmera, eds., Brno, Czech Republic, pp. 50-57.
[26] Michalewicz, Z., and Fogel, D. B., (2000), How to solve it : modern heuristics, Springer, Berlin.
[27] Coello Coello, C. A., (2002), Theoretical and numerical constraint-handling techniques used with
evolutionary algorithms: a survey of the state of the art, Computer Methods in Applied Mechanics
and Engineering, Vol. 191, No. 11-12, pp. 1245-1287.
[28] Jimenez, F., Verdegay, J. L., and Gomez-Skarmeta, A. F., (1999), Evolutionary techniques for
constrained multiobjective optimization problems, Proc. Proceedings of the workshop on multi-
criterion optimization using evolutionary methods held at genetic and evolutionary computation
conference (GECCO-1999), pp. 115-116.
[29] Pulido, G. T., and Coello, C. A. C., (2004), A constraint-handling mechanism for particle swarm
optimization, Proc. Proceedings of the 2004 Congress on Evolutionary Computation, Piscataway,
NJ, pp. 1396-1403.
[30] Zavala, A. E. M., Aguirre, A. H., and Villa Diharce, E. R., (2005), Constrained optimization via
Particle Evolutionary Swarm Optimization algorithm (PESO), Proc. GECCO 2005 - Genetic and
Evolutionary Computation Conference, Washington, DC, pp. 209-216.
[31] Mezura-Montes, E., Coello, C. A. C., and Tun-Morales, E. I., (2004), Simple feasibility rules and
differential evolution for constrained optimization, Proc. Lecture Notes in Artificial Intelligence
(Subseries of Lecture Notes in Computer Science), pp. 707-716.
[32] Barr, R. S., Golden, B. L., Kelly, J. P., Resende, M. G. C., and Stewart, W. R., (1995), Designing
and reporting on computational experiments with heuristic methods, Journal of Heuristics, Vol. 1,
No. 1, pp. 9-32.

27
[33] Hock, W., and Schittkowski, K., (1981), Test Examples for Nonlinear Programming Codes, Lecture
notes in economics and mathematical systems, Springer-Verlag New York, Secaucus.
[34] Himmelblau, D. M., (1972), Applied Nonlinear Programming, McGraw-Hill Book Company, New
York.
Appendix: 12 test functions
f1
Objective function: f(x) = x1 + x2 + x3
Constraints:
1 – 0.0025(x4 + x6) ≥ 0
1 – 0.0025(x5 + x7 – x4) ≥ 0
1 – 0.01(x8 – x5) ≥ 0
x1x6 – 833.33252x4 – 100x1 + 83333.333 ≥ 0
x2x7 – 1250x5 – x2x4 + 1250x4 ≥ 0
x3x8 – 1250000 – x3x5 + 2500x5 ≥ 0
100 ≤ x1 ≤ 10000
1000 ≤ xi ≤ 10000, i = 2, 3
10 ≤ xi ≤ 1000, i = 4,…, 8
Solution:
f(xmin) = 7049.330923
xmin = (579.3167, 1359.943, 5110.071, 182.0174, 295.5985, 217.9799, 286.4162, 395.5979)
f2
Objective function: f(x) = 1 + x1 + x2 + x3 + x4
Constraints:
0.0401 – 4/x1 – 2.25/x2 – 1/x3 – 0.25/x4 ≥ 0
0.010085 – 0.16/x1 – 0.36/x2 – 0.64/x3 – 0.64/x4 ≥ 0
0.001 ≤ xi ≤ 100000(5 – i), i = 1,…, 4
Solution:
f(xmin) = 727.67937
xmin = (193.4071, 179.5475, 185.0186, 168.7062)
f3
Objective function: f(x) = 0.2x3 – 0.8x1
Constraints:
x2 – exp(x1) ≥ 0
x3 – exp(x2) ≥ 0
0 ≤ x1 ≤ 100
0 ≤ x2 ≤ 100
0 ≤ x3 ≤ 10
Solution:
f(xmin) = 0.5181632741
xmin = (0.1841264879, 1.202167873, 3.327322322)
f4
Objective function: f(x) = 5.3578547x3
2
+ 0.8356891x1x5 + 37.293239x1 – 40792.141
Constraints:
92 ≥ 85.334407 + 0.0056858x2x5 + 0.0006262x1x4 – 0.0022053x3x5 ≥ 0
20 ≥ 80.51249 + 0.0071317x2x5 + 0.0029955x1x2 + 0.0021813x3
2
– 90 ≥ 0
5 ≥ 9.300961 + 0.0047026x3x5 + 0.0012547x1x3 + 0.0019085x3x4 – 20 ≥ 0
78 ≤ x1 ≤ 102
33 ≤ x2 ≤ 45
27 ≤ xi ≤ 45, i = 3, 4, 5

28
Solution:
f(xmin) = -30665.53867
xmin = (78, 33, 29.99526, 45, 36.77581)
f5
Objective function: f(x) = (x1 – 2)2
+ x2
2
Constraints:
(1 – x1)3
– x2 ≥ 0
0 ≤ x1 ≤ 1000
0 ≤ x2 ≤ 1000
Solution:
f(xmin) = 1
xmin = (1, 0)
f6
Objective function: f(x) = 5.04x1 + 0.035x2 + 10x3 + 3.36x5 – 0.063x4x7
Constraints:
g1(x) = 35.82 – 0.222x10 – 0.9x9 ≥ 0
g2(x) = –133 + 3x7 – 0.99x10 ≥ 0
g3(x) = –g1(x) + x9(1/0.9 – 0.9) ≥ 0
g4(x) = –g2(x) + (1/0.99 – 0.99)x10 ≥ 0
g5(x) = 1.12x1 + 0.13167x1x8 – 0.00667x1x8
2
– 0.99x4 ≥ 0
g6(x) = 57.425 + 1.098x8 – 0.038x8
2
+ 0.325x6 – 0.99x7 ≥ 0
g7(x) = –g5(x) + (1/0.99 – 0.99)x4 ≥ 0
g8(x) = –g6(x) + (1/0.99 – 0.99)x7 ≥ 0
g9(x) = 1.22x4 – x1 –x5 = 0
g10(x) = 98000x3/(x4x9 + 1000x3) – x6 = 0
g11(x) = (x2 + x5)/x1 – x8 = 0
0.00001 ≤ x1 ≤ 2000
0.00001 ≤ x2 ≤ 16000
0.00001 ≤ x3 ≤ 120
0.00001 ≤ x4 ≤ 5000
0.00001 ≤ x5 ≤ 2000
85 ≤ x6 ≤ 93
90 ≤ x7 ≤ 95
3 ≤ x8 ≤ 12
1.2 ≤ x9 ≤ 4
145 ≤ x10 ≤ 162
Solution:
f(xmin) = -1768.80696
xmin = (1698.096, 15818.73, 54.10228, 3031.226, 2000, 90.11537, 95, 10.49336, 1.561636, 153.53535)
f7
Objective function: f(x) = (x1 – 10)3
+ (x2 – 20)3
Constraints:
(x1 – 5)2
+ (x2 – 5)2
– 100 ≥ 0
–(x2 – 5)2
– (x1 – 6)2
+ 82.81 ≥ 0
13 ≤ x1 ≤ 100
0 ≤ x2 ≤ 100
Solution:
f(xmin) = -6961.81381
xmin = (14.095, 0.84296079)
f8
Objective function: f(x) = x1x4(x1 + x2 + x3) + x3
Constraints:

29
x1x2x3x4 – 25 ≥ 0
x1
2
+ x2
2
+ x3
2
+ x4
2
– 40 = 0
1 ≤ xi ≤ 5, i = 1,…, 4
Solution:
f(xmin) = 17.0140173
xmin = (1, 4.7429994, 3.8211503, 1.3794082)
f9
Objective function: f(x) = 3x1 + 1E-6x1
3
+ 2x2 + 2/3E-6x2
3
Constraints:
x4 – x3 + 0.55 ≥ 0
x3 – x4 + 0.55 ≥ 0
1000sin(-x3 – 0.25) + 1000sin(-x4 – 0.25) + 894.8 – x1 = 0
1000sin(x3 – 0.25) + 1000sin(x3 -x4 – 0.25) + 894.8 – x2 = 0
1000sin(x4 – 0.25) + 1000sin(x4 –x3 – 0.25) + 1294.8 = 0
0 ≤ xi ≤ 1200, i = 1, 2
-0.55 ≤ xi ≤ 0.55, i = 3, 4
Solution:
f(xmin) = 5126.4981
xmin = (679.9453, 1026.067, 0.1188764, -0.3962336)
f10
Objective function: f(x) = -75.196 + 3.8112x1 + 0.0020567x1
3
– 1.0345E-5x1
4
+ 6.8306x2 – 0.030234x1x2 +
1.28134E-3x2x1
2
+ 2.266E-7x1
4
x2 – 0.25645x2
2
+ 0.0034604x2
3
– 1.3514E-5x2
4
+ 28.106/(x2 + 1) +
5.2375E-6x1
2
x2
2
+ 6.3E-8x1
3
x2
2
– 7E-10x1
3
x2
3
– 3.405E-4x1x2
2
+ 1.6638E-6x1x2
3
+ 2.8673exp(0.0005x1x2)
– 3.5256E-5x1
3
x2 – 0.12694x1
2
Constraints:
x1x2 – 700 ≥ 0
x2 – x1
2
/125 ≥ 0
(x2 – 50)2
– 5(x1 – 55) ≥ 0
0 ≤ x1 ≤ 75
0 ≤ x2 ≤ 65
Solution:
f(xmin) = -7.804226324
xmin = (13.55010424, 51.66018129)
f11
Objective function: f(x) = exp(x1x2x3x4x5)
Constraints:
x1
2
+ x2
2
+ x3
2
+ x4
2
+ x5
2
– 10 = 0
x2x3 – 5x4x5 = 0
x1
3
+ x2
3
+ 1 = 0
-2.3 ≤ xi ≤ 2.3, i = 1, 2
-3.2 ≤ xi ≤ 3.2, i = 3, 4, 5
Solution:
f(xmin) = 0.0539498478
xmin = (-1.717143, 1.595709, 1.827247, -0.7636413, -0.763645)
f12
Objective function:
f(x) = ∑ ∑
= =
−
+
10
1
j
10
1
k
k
j
j
j )))
exp(x
ln(
x
)(c
exp(x
c1 = -6.089, c2 = -17.164, c3 = -34.054, c4 = -5.914, c5 = -24.721, c6 = -14.986, c7 = -24.100, c8 = -10.708, c9
= -26.662, c10 = -22.179
Constraints:
exp(x1) + 2exp(x2) + 2exp(x3) + exp(x6) + exp(x10) – 2 = 0

30
exp(x4) + 2exp(x5) + exp(x6) + exp(x7) – 1 = 0
exp(x3) + exp(x7) + exp(x8) + 2exp(x9) + exp(x10) – 1 = 0
-100 ≤ xi ≤ 100, i = 1,…, 10
Solution:
f(xmin) = -47.76109026
xmin = (-3.201212, -1.912060, -0.2444413, -6.537489, -0.7231524, -7.267738, -3.596711, -4.017769, -
3.287462, -2.335582)

A Comparison of Particle Swarm Optimization and Differential Evolution

More Related Content

Similar to A Comparison of Particle Swarm Optimization and Differential Evolution (20)

Recently uploaded (20)

A Comparison of Particle Swarm Optimization and Differential Evolution