Derivative Free Optimization
Download as ODP, PDF1 like470 views
The document discusses derivative-free optimization and evolutionary algorithms. It begins with an introduction to derivative-free optimization, explaining why it is useful when derivatives are unavailable or functions are noisy. Evolutionary algorithms are then discussed, including their fundamental elements like populations, selection, and variation operators. Specific evolutionary algorithms are presented, such as the estimation of distribution algorithm (EDA) and the (1+1)-ES algorithm with 1/5th success rule adaptation. The slides note that evolutionary algorithms are robust to noise and difficult optimization problems but are generally slower than derivative-based methods.
Derivative Free Optimization
- 1. DERIVATIVE-FREE OPTIMIZATION http://www.lri.fr/~teytaud/dfo.pdf (or Quentin's web page ?) Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège using also Slides from A. Auger
- 2. The next slide is the most important of all. Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 3. In case of trouble, Interrupt me. Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 4. In case of trouble, Interrupt me. Further discussion needed: - R82A, Montefiore institute - olivier.teytaud@inria.fr - or after the lessons (the 25 th , not the 18th) Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 6. I. Optimization and DFO II. Evolutionary algorithms III. From math. programming IV. Using machine learning V. Conclusions Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 7. Derivative-free optimization of f Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 8. Derivative-free optimization of f Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 9. Derivative-free optimization of f No gradient ! Only depends on the x's and f(x)'s Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 10. Derivative-free optimization of f Why derivative free optimization ?
- 11. Derivative-free optimization of f Why derivative free optimization ? Ok, it's slower
- 12. Derivative-free optimization of f Why derivative free optimization ? Ok, it's slower But sometimes you have no derivative
- 13. Derivative-free optimization of f Why derivative free optimization ? Ok, it's slower But sometimes you have no derivative It's simpler (by far) ==> less bugs
- 14. Derivative-free optimization of f Why derivative free optimization ? Ok, it's slower But sometimes you have no derivative It's simpler (by far) It's more robust (to noise, to strange functions...)
- 15. Derivative-free optimization of f Optimization algorithms ==> Newton optimization ? Why derivative free ==> Quasi-Newton (BFGS) Ok, it's slower But sometimes you have no derivative ==> Gradient descent It's simpler (by far) ==> ...robust (to noise, to strange functions...) It's more
- 16. Derivative-free optimization of f Optimization algorithms Why derivative free optimization ? Ok, it's slower Derivative-free optimization But sometimes you have no derivative (don't need gradients) It's simpler (by far) It's more robust (to noise, to strange functions...)
- 17. Derivative-free optimization of f Optimization algorithms Why derivative free optimization ? Derivative-free optimization Ok, it's slower But sometimes you have no derivative Comparison-based optimization (coming soon), It's simpler (by far)comparisons, just needing It's more robust (to noise, to strange functions...) incuding evolutionary algorithms
- 18. I. Optimization and DFO II. Evolutionary algorithms III. From math. programming IV. Using machine learning V. Conclusions Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 19. II. Evolutionary algorithms a. Fundamental elements b. Algorithms c. Math. analysis Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 20. Preliminaries: - Gaussian distribution - Multivariate Gaussian distribution - Non-isotropic Gaussian distribution - Markov chains ==> for theoretical analysis
- 21. Preliminaries: - Gaussian distribution - Multivariate Gaussian distribution - Non-isotropic Gaussian distribution - Markov chains
- 22. K exp( - p(x) ) with - p(x) a degree 2 polynomial (neg. dom coef) - K a normalization constant Preliminaries: - Gaussian distribution - Multivariate Gaussian distribution - Non-isotropic Gaussian distribution - Markov chains
- 23. K exp( - p(x) ) with - p(x) a degree 2 polynomial (neg. dom coef) - K a normalization constant Translation of the Preliminaries:Gaussian Sze of the - Gaussian distribution Gaussian - Multivariate Gaussian distribution - Non-isotropic Gaussian distribution - Markov chains
- 24. Preliminaries: - Gaussian distribution - Multivariate Gaussian distribution - Non-isotropic Gaussian distribution
- 25. Preliminaries: Isotropic case: - Gaussian distribution - Multivariate Gaussian distribution||2 /22) ==> general case: density = K exp( - || x - ==> level sets are rotationally invariant - Non-isotropic Gaussian distribution ==> completely defined by and - Markov chains (do you understand why K is fixed by ?) ==> “isotropic” Gaussian
- 26. Preliminaries: - Gaussian distribution - Multivariate Gaussian distribution - Non-isotropic Gaussian distribution
- 27. Step-size different on each axis K exp( - p(x) ) with - p(x) a quadratic form (--> + infinity) - K a normalization constant
- 28. Notions that we will see: - Evolutionary algorithm - Cross-over - Truncation selection / roulette wheel - Linear / log-linear convergence - Estimation of Distribution Algorithm - EMNA - Self-adaptation - (1+1)-ES with 1/5th rule - Voronoi representation - Non-isotropy
- 29. Comparison-based optimization Observation: we want robustness w.r.t that: is comparison-based if Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 29
- 30. Comparison-based optimization yi=f(xi) is comparison-based if Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 30
- 31. Population-based comparison-based algorithms ? X(1)=( x(1,1),x(1,2),...,x(1,) ) = Opt() X(2)=( x(2,1),x(2,2),...,x(2,) ) = Opt(x(1), signs of diff) … … ... x(n)=( x(n,1),x(n,2),...,x(n,) ) = Opt(x(n-1), signs of diff) ==> let's write it for =2. Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 31
- 32. Population-based comparison-based algorithms ? x(1)=(x(1,1),x(1,2)) = Opt() x(2)=(x(2,1),x(2,2)) = Opt(x(1), sign(y(1,1)-y(1,2)) ) … … ... x(n)=(x(n,1),x(n,2)) = Opt(x(n-1), sign(y(n-1,1)-y(n-1,2)) with y(i,j) = f ( x(i,j) ) Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 32
- 33. Population-based comparison-based algorithms ? Abstract notations: x(i) is a population x(1) = Opt() x(2) = Opt(x(1), sign(y(1,1)-y(1,2)) ) … … ... x(n) = Opt(x(n-1), sign(y(n-1,1)-y(n-1,2)) Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 33
- 34. Population-based comparison-based algorithms ? Abstract notations: x(i) is a population, I(i) is an internal state of the algorithm. x(1),I(1) = Opt() x(2),I(2) = Opt(x(1), sign(y(1,1)-y(1,2)), I(1) ) … … ... x(n),I(n) = Opt(x(n-1),sign(y(n-1,1)-y(n-1,2) ,I(n-1)) Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 34
- 35. Population-based comparison-based algorithms ? Abstract notations: x(i) is a population, I(i) is an internal state of the algorithm. x(1),I(1) = Opt() x(2),I(2) = Opt(x(1), (1), I(1) ) … … ... x(n),I(n) = Opt(x(n-1),(n-1) ,I(n-1)) Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 35
- 36. Comparison-based optimization ==> Same behavior on many functions is comparison-based if Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 36
- 37. Comparison-based optimization ==> Same behavior on many functions is comparison-based if Quasi-Newton methods very poor on this. Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 37
- 38. Why comparison-based algorithms ? ==> more robust ==> this can be mathematically formalized: comparison-based opt. are slow ( d log ||xn-x*||/n ~ constant) but robust (optimal for some worst case analysis) Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 39. II. Evolutionary algorithms a. Fundamental elements b. Algorithms c. Math. analysis Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 40. Parameters: Generate points around x x, ( x + N where N is a standard Gaussian) o f an egy ema Strat c sch ution Basi Evol
- 41. Parameters: Generate points around x x, ( x + N where N is a standard Gaussian) o f an egy Compute their fitness values ema Strat c sch ution Basi Evol
- 42. Parameters: Generate points around x x, ( x + N where N is a standard Gaussian) o f an egy Compute their fitness values ema Strat c sch ution Select the best Basi Evol
- 43. Parameters: Generate points around x x, ( x + N where N is a standard Gaussian) o f an egy Compute their fitness values ema Strat c sch ution Select the best Basi Evol Let x = average of these best
- 44. Parameters: Generate points around x x, ( x + N where N is a standard Gaussian) o f an egy Compute their fitness values ema Strat c sch ution Select the best Basi Evol Let x = average of these best
- 45. Parameters: Generate points around x x, ( x + N where N is a standard Gaussian) llel para Compute their fitness values Multi-cores, sly Clusters, Grids... ou Select the best Obvi Let x = average of these best
- 46. Parameters: Generate points around x x, ( x + N where N is a standard Gaussian) llel para Compute their fitness values sly ple. ou Select the best ly sim Obvi Real Let x = average of these best
- 47. Parameters: Generate points around x x, ( x + N where N is a standard Gaussian) llel para Not a negligible advantage. Compute their fitness values When I accessed, for the 1st time, sly to a crucial industrial ple. code of an important ou Select the best company, I believed ly sim Obvi that it would be Real clean and bug free. Let x = average of these best (I was young)
- 48. Parameters: Generate 1 point x' around x x, ( x + N where N is a standard Gaussian) Compute its fitness value ) - ES le Keep the best (x or x'). ru (1 +1 1/5 th x=best(x,x') The with =2 if x' best =0.84 otherwise
- 49. This is x...
- 50. I generate =6 points
- 51. I select the =3 best points
- 52. x=average of these =3 best points
- 53. Ok. Choosing an initial x is as in any algorithm. But how do I choose sigma ?
- 54. Ok. Choosing x is as in any algorithm. But how do I choose sigma ? Sometimes by human guess. But for large number of iterations, there is better.
- 58. log || xn – x* || ~ - C n
- 59. Usually termed “linear convergence”, ==> but it's in log-scale. log || xn – x* || ~ - C n
- 60. Examples of evolutionary algorithms Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 60
- 61. Estimation of Multivariate Normal Algorithm Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 61
- 62. Estimation of Multivariate Normal Algorithm Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 62
- 63. Estimation of Multivariate Normal Algorithm Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 63
- 64. Estimation of Multivariate Normal Algorithm Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 64
- 65. EMNA is usually non-isotropic Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 65
- 66. EMNA is usually non-isotropic Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 66
- 67. Self-adaptation (works in many frameworks) Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 67
- 68. Self-adaptation (works in many frameworks) Can be used for non-isotropic multivariate Gaussian distributions. Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 68
- 69. Let's generalize. We have seen algorithms which work as follows: - we keep one search point in memory (and one step-size) - we generate individuals - we evaluate these individuals - we regenerate a search point and a step-size Maybe we could keep more than one search point ?
- 70. Let's generalize. We have seen algorithms which work as follows: - we keep one search point in memory (and one step-size) points ==> mu search - we generate individuals - we evaluate thesegenerated individuals ==> lambda individuals - we regenerate a search point and a step-size Maybe we could keep more than one search point ?
- 71. Parameters: Generate points x1,...,x around x1,...,x e.g. each x randomly generated from two points llel para Compute their fitness values sly ple. ou Select the best ly sim Obvi Real Don't average...
- 72. Generate points around x1,...,x e.g. each x randomly generated from two points
- 73. Generate points around x1,...,x e.g. each x randomly generated This is a from two points cross-over
- 74. Generate points around x1,...,x e.g. each x randomly generated This is a from two points cross-over Example of procedure for generating a point: - Randomly draw k parents x1,...,xk (truncation selection: randomly in selected individuals) - For generating the ith coordinate of new individual z: u=random(1,k) z(i) = x(u)i
- 75. Let's summarize: We have seen a general scheme for optimization: - generate a population (e.g. from some distribution, or from a set of search points) - select the best = new search points ==> Small difference between an Evolutionary Algorithm (EA) and an Estimation of Distribution Algorithm (EDA). ==> Some EA (older than the EDA acronym) are EDAs.
- 76. Let's summarize: We have seen a general scheme for optimization: - generate a population (e.g. from some distribution, or from a set of search points) - select the best = new search points EDA EA ==> Small difference between an Evolutionary Algorithm (EA) and an Estimation of Distribution Algorithm (EDA). ==> Some EA (older than the EDA acronym) are EDAs.
- 77. Gives a lot freedom: - choose your representation and operators (depending on the problem) - if you have a step-size, choose adaptation rule - choose your population-size (depending on your computer/grid ) - choose (carefully) e.g. min(dimension, /4)
- 78. Gives a lot freedom: - choose your operators (depending on the problem) - if you have a step-size, choose adaptation rule - choose your population-size (depending on your computer/grid ) - choose (carefully) e.g. min(dimension, /4) Can handle strange things: - optimize a physical structure ? - structure represented as a Voronoi - cross-over makes sense, benefits from local structure - not so many algorithms can work on that
- 79. Voronoi representation: - a family of points
- 80. Voronoi representation: - a family of points
- 81. Voronoi representation: - a family of points - their labels
- 82. Voronoi representation: - a family of points - their labels ==> cross-over makes sense ==> you can optimize a shape
- 83. Voronoi representation: - a family of points - their labels ==> cross-over makes sense ==> you can optimize a shape ==> not that mathematical; but really useful Mutations: each label is changed with proba 1/n Cross-over: each point/label is randomly drawn from one of the two parents
- 84. Voronoi representation: - a family of points - their labels ==> cross-over makes sense ==> you can optimize a shape ==> not that mathematical; but really useful Mutations: each label is changed with proba 1/n Cross-over: randomly pick one split in the representation: - left part from parent 1 - right part from parent 2 ==> related to biology
- 85. Gives a lot freedom: - choose your operators (depending on the problem) - if you have a step-size, choose adaptation rule - choose your population-size (depending on your computer/grid ) - choose (carefully) e.g. min(dimension, /4) Can handle strange things: - optimize a physical structure ? - structure represented as a Voronoi - cross-over makes sense, benefits from local structure - not so many algorithms can work on that
- 86. II. Evolutionary algorithms a. Fundamental elements b. Algorithms c. Math. Analysis Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 87. Consider the (1+1)-ES. x(n) = x(n-1) or x(n-1) + (n-1)N We want to maximize: - E log || x(n) - f* || Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 88. Consider the (1+1)-ES. x(n) = x(n-1) or x(n-1) + (n-1)N We want to maximize: - E log || x(n) - f* || -------------------------- - E log || x(n-1) – f* || Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 89. Consider the (1+1)-ES. x(n) = x(n-1) or x(n-1) + (n-1)N We don't know f*. We want to maximize: How can we optimize this ? - E log || x(n) - f* || We will observe -------------------------- the acceptance rate, - E log || x(n-1) – f* || and we will deduce if Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège is too large or too small..
- 90. - E log || x(n) - f* || ON THE NORM FUNCTION -------------------------- - E log || x(n-1) – f* || Rejected Accepted mutations mutations Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 91. - E log || x(n) - f* || For each step-size, -------------------------- evaluate this “expected progress rate” - E log || x(n-1) – f* || and evaluate “P(acceptance)” Rejected Accepted mutations mutations Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 92. Progress rate Rejected mutations Acceptance rate
- 93. Progress rate We want to be here! Rejected We observe mutations (approximately) this variable Acceptance rate
- 94. Progress rate Rejected mutations Big Acceptance rate step-size
- 95. Progress rate Rejected mutations Small step-size Acceptance rate
- 96. Progress rate Rejected Small acceptance rate mutations ==> decrease sigma Acceptance rate
- 97. Progress rate Rejected Big acceptance rate mutations ==> increase sigma Acceptance rate
- 98. th 1/5 rule Based on maths showing that good step-size <==> success rate < 1/5 Auger, Fournier, Hansen, Rolet, Teytaud, Teytaud parallel evolution 98
- 99. I. Optimization and DFO II. Evolutionary algorithms III. From math. programming IV. Using machine learning V. Conclusions Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 100. III. From math. programming ==>pattern search method Comparison with ES: - code more complicated - same rate - deterministic - less robust Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 101. III. From math. programming Also: - Nelder-Mead algorithm (similar to pattern search, better constant in the rate) Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 102. III. From math. programming Also: - Nelder-Mead algorithm (similar to pattern search, better constant in the rate) - NEWUOA (using value functions and not only comparisons) Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 103. I. Optimization and DFO II. Evolutionary algorithms III. From math. programming IV. Using machine learning V. Conclusions Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 104. IV. Using machine learning What if computing f takes days ? ==> parallelism ==> and “learn” an approximation of f Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 105. IV. Using machine learning Statistical tools: f ' (x) = approximation ( x, x1,f(x1), x2,f(x2), … , xn,f(xn)) y(n+1) = f ' (x(n+1) ) e.g. f' = quadratic function closest to f on the x(i)'s.
- 106. IV. Using machine learning ==> keyword “surrogate models” ==> use f' instead of f ==> periodically, re-use the real f
- 107. I. Optimization and DFO II. Evolutionary algorithms III. From math. programming IV. Using machine learning V. Conclusions Olivier Teytaud Inria Tao, en visite dans la belle ville de Liège
- 108. Derivative free optimization is fun. ==> nice maths ==> nice applications + easily parallel algorithms ==> can handle really complicated domains (mixed continuous / integer, optimization on sets of programs) Yet, often suboptimal on highly structured problems (when BFGS is easy to use, thanks to fast gradients)
- 109. Keywords, readings ==> cross-entropy (so close to evolution strategies) ==> genetic programming (evolutionary algorithms for automatically building programs) ==> H.-G. Beyer's book on ES = good starting point ==> many resources on the web ==> keep in mind that representation / operators are often the key ==> we only considered isotropic algorithms; sometimes not a good idea at all