Nelder Mead Search Algorithm

Nelder Mead Search Algorithm Presented By: Ashish Khetan IIT Guwahati, IndiaSupervisor: Prof. Dr. Ian Cloete President IU in Germany

Numerical method for minimizing an objective function in a multi dimensional space Is it like the linear programming ? Answer is NO !!!Only applicable to unconstrainedproblems.Other names – Downhill simplex method or amoeba method. 2What does algorithm do ?

SimplexAffine space Convex hull Polytope 3Prerequisites to learn the algorithm

An affine space is what is left of a vector space after you've forgotten which point is the origin.An affine subspace of a vector space V is a subset closed under affine combinations of vectors in the space. For example, the set is an affine space, where {vi}i is a family of vectors in V.the sum of the coefficients is 14Affine space

The convex hull for a set of points X in a real vector space V is the minimal convex set containing X.For planar objects, it can be easily visualized by imagining an elastic band stretched open to encompass the given object; when released, it will assume the shape of the required convex hull.Same concept can be extended to multi-dimensional space. 5Convex hull

Polytope means, first, the generalization to any dimension of polygon in two dimensions, polyhedron in three dimensions, and polychoron in four dimensions.One special kind of polytope is a convex polytope, which is the convex hull of a finite set of points. A convex polytope can also be represented as the intersection of half-spaces. This intersection can be written as the matrix inequality:6Polytope

where A is an m by n matrix, m being the number of bounding half-spaces and n being the number of dimensions of the affine space Rn in which the polytope iscontained; and b is an m by 1 column vector.7Polytope continued….

A polytope of a N+1 vertices in N dimensions. A line segment on a line. A triangle on a plane.A tetrahedron in three dimensional space and so forth. 8Simplex

For a function or system withN variables, algorithm starts with N+1 points in a N dimensional space, defining an initial simplex. Take a random initial starting point Po and then the other N points can be taken by - Pi = Po + λeiwhere eiare N unit vectors. λ is a constant, and its value depends upon the problem’s characteristic length scale. Different λ’scan be chosen for each vector direction. 9Core of the algorithm

Calculate the function value(for a system calculate the value of objective function) at each of the vertices of the simplex. Sort the points in order of the value of the function at that point. Replace the worst point with a point reflected through the centroid of the remaining N points. If this point is better than the best current point, then we can try stretching out/in along this line. 10How the algorithm works ?

If this new point is not much better than the previous value, then we are steeping across a valley, so we shrink the simplex towards the best point. It may get stuck in a rut. Then restart the algorithm with a new simplex starting at the current bets value. Many variations exist depending upon the actual nature of the problem being solved. 11Continued……

Sort the simplex vertices according to the function value at that point. Compute the centroid Xo using all points except Xn+1 , the worst point. Compute a reflection Now there are three possibilities. 12Steps

1. , use Xr and reject Xn+1 , go to step 1. 2. , then computeXe = Xo + ρ(Xr - Xo ) (expansion) if , then use Xe else use Xr , reject Xn+1 , go to step 1. 13Continue….

3.(contraction) If , then use Xc and reject Xn+1 , go to step 1. Else …..Shrink step- compute new N vertices keeping only the best one, X1. , go to step 1. 14Continue…..

Reflection coefficient – α =1.Expansion coefficient – ρ =2.Contraction coefficient – γ = ½. Shrink coefficient – σ =½.15Standard values of the algorithm parametrs

It can find the optimum value of any numbers of parameters involved in any system.Optimum value of parameters means value of parameters while the value of objective function is minimum.It has nothing to do with the system, what all it does is iteratively runs the system with different set of values of the parameters and eventually terminates at the minimum value of the objective function providing the optimum value of parameters. 16Upside of the algorithm

It can find only local minimums. The local minima that the algorithm finds entirely depends upon the initial simplex starting point. To find the different local minimums, algorithm must be started with the different and appropriate initial simplex guess. For a black box system where there is no knowledge of number of minimums, only hit and trial method works to find all the local minimums. 17Downside of the algorithm

http://math.fullerton.edu/mathews/n2003/neldermead/NelderMeadMod/Links/NelderMeadMod_lnk_5.htmlThe above link provides the best illustrative example of this algorithm. 18Illustrative example

In the same folder find a file Nelder Mead.java, it runs over a N dimensional algebraic function and takes function as a input from user and finds it’s one of the local minimum depending upon the initial simplex starting point guess provided by the user. The same code can be modified to run over a system and to find the optimum value of parameters for a given objective function. 19JAVA code

Numerical Recipes in C , The Art of Scientific Computing, Second Edition. http://en.wikipedia.org/wiki/Affine_spacehttp://en.wikipedia.org/wiki/Convex_hullhttp://en.wikipedia.org/wiki/Polytopehttp://en.wikipedia.org/wiki/Nelder-Mead_method20References

Nelder Mead Search Algorithm

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Nelder Mead Search Algorithm (20)

More from Ashish Khetan (7)

Recently uploaded (20)

Nelder Mead Search Algorithm