UMDAs for Dynamic Optimization Problems
Download as PPT, PDF2 likes274 views
The document proposes an update strategy called Reinforcement/Evaporation (RE) for Univariate Marginal Distribution Algorithms (UMDAs) to better handle Dynamic Optimization Problems (DOPs). RE is inspired by the update equations used in Ant Colony Optimization (ACO) algorithms. The RE strategy maintains diversity in the population to avoid full convergence, which is unsuitable for dynamic problems. Experimental results show that RE outperforms other diversity-handling techniques and allows the UMDA to better track optima in changing environments.
UMDAs for Dynamic Optimization Problems
- 1. UMDAs for Dynamic Optimization Problems Carlos Fernandes 1,2 Claudio Lima 3 Agostinho C. Rosa 1 1 LaSEEB-ISR-IST, Technical Univ. of Lisbon (IST), Portugal 2 Department of Architecture and Computer Technology, University of Granada, Spain 3 Informatics Laboratory, University of Algarve, Portugal GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 2. Motivation and Objectives Design an update strategy for UMDAs to deal with Dynamic Optimization Problems (DOPs), based on ACO. Problem: full convergence. When solving DOPs, finding the optima is not the main task; the algorithm must track the optima. Full convergence (without ways to escape it) is not suitable for DOPs (even if the population converges to the global optimum) Solution: delay or avoid full convergence. Change probability distribution. Maintain probability distribution except when close to 0 and 1 Combination of strategies GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 3. Motivation and Objectives Proposal: Update strategy for UMDA based on ACO equations. Effects: Change probability distribution. Why? Reinforcement/evaporation equations allow us to control the convergence speed of the algorithm ACO algorithms build solutions by travelling trough the nodes of a (combinatorial) problem. (TSP, for instance…) After the evaluation of the solutions, edges that belong to good solutions are reinforced (pheromone) Each generation, pheromone in all edges is evaporated – diversity, mutation. In the following generation, solutions/paths are chosen according to the amount of pheromone in each connection between nodes. GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 4. Motivation and Objectives ACO Univariate EDA initialize pheromone in all edges repeat sample N solutions evaluate solutions update pheromone until stop criterion initialize probability model repeat sample N solutions evaluate solutions update model parameters until stop criterion GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 5. Motivation and Objectives The Binary Ant Algorithm (presented in GECCO07). Based on Ant Colony Optimization (ACO), BAA builds pheromone trails between binary variables. BAA is a kind of Estimation of Distribution Algorithm (EDA) There are similarities between EDAs and ACO. The pheromone trails are similar to the probability models, and reinforcement/evaporation is ACO’s update strategy. 0 0 0 1 1 1 0 1 0 1 Solution Dimension BAA GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 6. Reducing Diversity Loss Branke, J., Lode, C., and Shapiro, J. 2007. Adressing sampling errors and diversity loss in UMDA . Proceedings of the 2007 Genetic and Evolutionary Computation Conference, ACM, 508-515. Permutation Sampling: reduces loss due to sampling; used in all the experiments Loss Correction (LC) Laplace Correction Iterated Laplace Correction (iLaplace) Boundary Correction: changes probability distribution near 0 and 1 Changes probability distribution GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 7. Reducing Diversity Loss GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 8. UMDA Set γ i ← 1/2 for all i = 1 . . .L; repeat Sample N strings according to make a population D . Generate a new population Ds from D by selecting the f × N fittest strings. for i = 1 to L do update model: end for until stop criterion Replace by ACO-like equations GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 9. Reinforcement/Evaporation (RE) Update Set γ i ← 1/2 for all i = 1 . . . L ; Set ← 0 for all i = 1 . . . L ; Set α and β repeat sample N strings according to make a population D . generate new population D s by selecting the f × N fittest strings. update pheromone evaporate for i = 1 to L do update model end for until stop criterion met β = 1 + + If α = 1 and β = 1, we have the standard update strategy 1 0 1 1… GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 10. Results – Diversity Loss in Flat Landscape β = 1; N = 20; f = 0.5; L = 100 α = 1; N = 20; f = 0.5; L = 100 GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 11. Test Set Severity of change: This criterion establishes how strongly the problem is changing Speed of change: This criterion establishes how often the environment changes Yang and Yao’s dynamic problems generator* By using a binary mask, dynamic environments are created by applying the mask to each solution before its evaluation. Severity of change is controlled by setting the number of 1’s in the mask. Speed of change is controlled by defining the number of generations between the application of a different mask. *Yang, S. and Yao, X. 2005. Experimental study on PBIL algorithms for dynamic optimization problems . Soft Computing 9(11), 815-834. GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 12. Test Set Functions Onemax Royal Road Speed was set to 𝜏 = 10, 100 (generations) Severity was set to ρ = 0.05, 0.6 and 0.95 6 different scenarios Performance is measured by the mean best-of-generation values, i.e., best fitness averaged over all generations, and then over all runs 30 runs for each configuration Compared RE with Loss, Laplace, iterated Laplace and Boundary Correction GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 13. Results – RE Parameters β = 1 β = 0.5 Royal Road Decreasing β improves performance when speed is high and severity is low β = 1 and α = 1 is standard UMDA’s update strategy GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 14. Results Comparing strategies that do not avoid full convergence RE1: α = 0.6, β = 1 RE2: α = 1, β = 0.5 GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs” Onemax Royal Road τ ρ 10 0.05 10 0.6 10 0.95 100 0.05 100 0.6 100 0.95 10 0.05 10 0.6 10 0.95 100 0.05 100 0.6 100 0.95 1 2 RE1 iLap − + + − − − − + ~ ~ ~ ~ RE1 LC + + + − − − + + + + + + RE2 iLap − + + ~ ~ + + + + + ~ ~ RE2 LC ~ + + ~ ~ + + + + + + + iLap LC + + + ~ ~ ~ + + + + + +
- 15. Results Table 2. Statistical analysis of the results in table 1. Avoiding full convergence RE1: α = 0.8, β = 1 RE2: α = 0.9, β = 0.5 *Laplace performs well when compared to iL+BC, LC+BC and BC GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs” Onemax Royal Road τ ρ 10 0.05 10 0.6 10 0.95 100 0.05 100 0.6 100 0.95 10 0.05 10 0.6 10 0.95 100 0.05 100 0.6 100 0.95 1 2 RE1+BC iLap+BC ~ + + ~ + + + + + + + + RE1+BC Laplace − + + − + + + + ~ + + + RE1+BC LC+BC − + + ~ + + + + ~ + + + RE1+BC BC − + + ~ + + + + + + + + RE2+BC iLap+BC − + + − − − + + ~ + + + RE2+BC Laplace − + + − − − ~ + ~ + + + RE2+BC LC+BC − + + − − − ~ ~ ~ ~ − ~ RE2+BC BC − + + − − − + + ~ + + +
- 16. Results Dynamic Royal Road with 𝜏 = 100. β = 1 ( α = 1 curves correspond to the standard UMDA update strategy) ρ = 0.05 ρ = 0.6 ρ = 0.9 RE GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 17. Results Table 2. Statistical analysis of the results in table 1. Dynamic Royal Road with 𝜏 = 100. α = 0.8, β = 1 RE Laplace GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 18. Conclusions RE is capable of outperforming other diversity loss correction techniques RE performs well when compared to Loss Correction and Iterated Laplace Correction RE with boundary correction outperforms other strategies in a wide range of scenarios Laplace Correction attains better results than other techniques (except RE) Diversity of the UMDA with RE may be controlled by α and β parameters RE seems to work well with α between 0.6 and 0.9, depending on β and depending if we hybridize it with Boundary Correction. GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 19. Conclusions RE strategy works well without needing to know when the fitness changes. There DOPs with changes that are not detectable (or that are too costly to detect). GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”
- 20. Future work In-depth study of the effects of parameter values on performance. Understand how α and β affect UMDA’s behavior. Tests on dynamic trap functions. Extend the strategy to other EDAs. GECCO08 - Atlanta Fernandes, Lima and Rosa – “UMDAs for DOPs”