Elementary Landscape Decomposition of Combinatorial Optimization Problems
- 1. Background on Conclusions Introduction Applications Landscapes & Future Work Elementary Landscape Decomposition of Combinatorial Optimization Problems Francisco Chicano Work in collaboration with L. Darrell Whitley Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 1 / 17
- 2. Background on Conclusions Introduction Applications Landscapes & Future Work Motivation Motivation • Landscapes’ theory is a tool for analyzing optimization problems • Peter F. Stadler is one of the main supporters of the theory • Applications in Chemistry, Physics, Biology and Combinatorial Optimization • Central idea: study the search space to obtain information • Better understanding of the problem • Predict algorithmic performance • Improve search algorithms Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 2 / 17
- 3. Background on Conclusions Introduction Applications Landscapes & Future Work Landscape Definition Elementary Landscapes Landscape decomposition Landscape Definition • A landscape is a triple (X,N, f) where X is the solution space The pair (X,N) is called N is the neighbourhood operator configuration space f is the objective function • The neighbourhood operator is a function s0 5 N: X →P(X) 2 s1 s3 2 • Solution y is neighbour of x if y ∈ N(x) s2 3 s5 1 • Regular and symmetric neighbourhoods 0 s7 s4 4 • d=|N(x)| ∀x∈X s6 0 • y ∈ N(x) ⇔ x ∈ N(y) 6 s8 s9 7 • Objective function f: X →R (or N, Z, Q) Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 3 / 17
- 4. Background on Conclusions Introduction Applications Landscapes & Future Work Landscape Definition Elementary Landscapes Landscape decomposition Elementary Landscapes: Formal Definition • An elementary function is an eigenvector of the graph Laplacian (plus constant) Adjacency matrix Degree matrix s0 • Graph Laplacian: s1 s3 s2 Depends on the s5 configuration space s7 s4 • Elementary function: eigenvector of Δ (plus constant) s6 s8 s9 Eigenvalue Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 4 / 17
- 5. Background on Conclusions Introduction Applications Landscapes & Future Work Landscape Definition Elementary Landscapes Landscape decomposition Elementary Landscapes: Characterizations • An elementary landscape is a landscape for which Depend on the problem/instance where Linear relationship • Grover’s wave equation Characteristic constant: k= - λ Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 5 / 17
- 6. Background on Conclusions Introduction Applications Landscapes & Future Work Landscape Definition Elementary Landscapes Landscape decomposition Elementary Landscapes: Properties • Some properties of elementary landscapes are the following where • Local maxima and minima Local maxima Local minima X Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 6 / 17
- 7. Background on Conclusions Introduction Applications Landscapes & Future Work Landscape Definition Elementary Landscapes Landscape decomposition Elementary Landscapes: Examples Problem Neighbourhood d k 2-opt n(n-3)/2 n-1 Symmetric TSP swap two cities n(n-1)/2 2(n-1) inversions n(n-1)/2 n(n+1)/2 Antisymmetric TSP swap two cities n(n-1)/2 2n Graph α-Coloring recolor 1 vertex (α-1)n 2α Graph Matching swap two elements n(n-1)/2 2(n-1) Graph Bipartitioning Johnson graph n2/4 2(n-1) NEAS bit-flip n 4 Max Cut bit-flip n 4 Weight Partition bit-flip n 4 Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 7 / 17
- 8. Background on Conclusions Introduction Applications Landscapes & Future Work Landscape Definition Elementary Landscapes Landscape decomposition Landscape Decomposition • What if the landscape is not elementary? • Any landscape can be written as the sum of elementary landscapes X X X • There exists a set of eigenfunctions of Δ that form a basis of the function space (Fourier basis) e2 Non-elementary function Elementary functions f Elementary components of f (from the Fourier basis) e1 Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 8 / 17
- 9. Background on Conclusions Introduction Applications Landscapes & Future Work Landscape Definition Elementary Landscapes Landscape decomposition Landscape Decomposition: Walsh Functions • If X is the set of binary strings of length n and N is the bit-flip neighbourhood then a Fourier basis is where Bitwise AND Bit count function (number of ones) • These functions are known as Walsh Functions • The function with subindex w is elementary with k=2 bc (w) • In general, decomposing a landscape is not a trivial task → methodology required Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 9 / 17
- 10. Background on Conclusions Introduction Applications Landscapes & Future Work Landscape Definition Elementary Landscapes Landscape decomposition Landscape Decomposition: Examples Problem Neighbourhood d Components inversions n(n-1)/2 2 General TSP swap two cities n(n-1)/2 2 QAP swap two elements n(n-1)/2 3 Frequency Assignment change 1 frequency (α-1)n 2 Subset Sum Problem bit-flip n 2 MAX k-SAT bit-flip n k NK-landscapes bit-flip n k+1 max. nb. of Radio Network Design bit-flip n reachable antennae Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 10 / 17
- 11. Background on Conclusions Introduction Applications Landscapes & Future Work Selection Strategy Autocorrelation New Selection Strategy • Selection operators usually take into account the fitness value of the individuals avg avg Minimizing Neighbourhoods X • We can improve the selection operator by selecting the individuals according to the average value in their neighbourhoods Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 11 / 17
- 12. Background on Conclusions Introduction Applications Landscapes & Future Work Selection Strategy Autocorrelation New Selection Strategy • In elementary landscapes the traditional and the new operator are the same! Recall that... • However, they are not the same in non-elementary landscapes. If we have n elementary components, then: Elementary components • The new selection strategy could be useful for plateaus Minimizing avg avg X Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 12 / 17
- 13. Background on Conclusions Introduction Applications Landscapes & Future Work Selection Operator Autocorrelation Autocorrelation • Let {x0, x1, ...} a simple random walk on the configuration space where xi+1∈N(xi) • The random walk induces a time series {f(x0), f(x1), ...} on a landscape. • The autocorrelation function is defined as: s0 5 2 2 s1 s3 3 s2 1 s5 0 • The autocorrelation length is defined as: s7 4 s4 0 s6 6 s8 s9 7 • Autocorrelation length conjecture: The number of local optima in a search space is roughly Solutions reached from x0 after l moves Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 13 / 17
- 14. Background on Conclusions Introduction Applications Landscapes & Future Work Selection Operator Autocorrelation Autocorrelation Length Conjecture • The higher the value of l the smaller the number of local optima and the better the performance of a local search method Angel, Zissimopoulos. Theoretical • l is a measure of the ruggedness of a landscape Computer Science 264:159-172 Nb. steps (config 1) Nb. steps (config 2) Length Ruggedness % rel. error nb. steps % rel. error nb. steps 10 ≤ ζ < 20 0.2 50500 0.1 101395 20 ≤ ζ < 30 0.3 53300 0.2 106890 30 ≤ ζ < 40 0.3 58700 0.2 118760 40 ≤ ζ < 50 0.5 62700 0.3 126395 50 ≤ ζ < 60 0.7 66100 0.4 133055 60 ≤ ζ < 70 1.0 75300 0.6 151870 70 ≤ ζ < 80 1.3 76800 1.0 155230 80 ≤ ζ < 90 1.9 79700 1.4 159840 90 ≤ ζ < 100 2.0 82400 1.8 165610 Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 14 / 17
- 15. Background on Conclusions Introduction Applications Landscapes & Future Work Selection Operator Autocorrelation Autocorrelation and Landscapes • If f is a sum of elementary landscapes: Fourier coefficients • For elementary landscapes: • Using the landscape decomposition we can determine a priori the performance of a local search method Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 15 / 17
- 16. Background on Conclusions Introduction Applications Landscapes & Future Work Conclusions & Future Work Conclusions & Future Work Conclusions • Elementary landscape decomposition is a useful tool to understand a problem • The decomposition can be used to design new operators • We can exactly determine the autocorrelation functions • It is not easy to find a decomposition in the general case Future Work • Methodology for landscape decomposition • Search for additional applications of landscapes’ theory in EAs • Design new operators and search methods based on landscapes’ information Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 16 / 17
- 17. Elementary Landscape Decomposition of Combinatorial Optimization Problems Thanks for your attention !!! Evolutionary Algorithms - Challenges in Theory and Practice, Bourdeaux, March 2010 17 / 17