Simplex Algorithm

Muhammad Kashif Nawaz
MS(CS) Batch # 3
Kashif.rpb@gmail.com

Contents Overview
• AGENDA
– Introduction
• KEY TERMINOLOGIES
• SIMPLEX ALGORITHM
• Software Simulation
• SIMPLEX ALGORITHM VARIATIONS
• REFERENCES
6/3/2014 Simplex Algorithm 2

Introduction(Background)
• Linear programming is the process of taking various linear inequalities
relating to some situation, and finding the "best" value obtainable under
those conditions.
• Linear programming is part of a very important area of mathematics called
optimization techniques.
• The graphical method is useful only for problems involving two decision
variables and relatively few problem constraints.

Introduction
• Question:
– Can we solve all LP problems using graphical approach ?
• NO
– WHY ???
» Real life systems can have dozens or hundreds of variables, or
more.
• What happens when we need more decision variables and more problem
constraints?
– Thus we need another systematic approach to solve the LP Problem.
– Simplex Algorithm

Introduction
• Simplex method which was developed by George B. DANTZIG (1914-2005)
in 1947.
• The most popular method used for the solution of Linear Programming
Problems (LPP) is the simplex method.
• The journal Computing in Science and Engineering listed it as one of the
top 10 algorithms of the twentieth century.
• Simplex Algorithm can solve an equation for many variables (dimensions).

Key Terminologies
• Standard Form & Slack Variable
• Basic & Non basic Variables
• Simple Tableau
• Pivoting

Standard Maximization Problems in
Standard Form
• A linear programming problem is said to be a standard maximization
problem in standard form if its mathematical model is of the following
form:
• Maximize the objective function
• Subject to problem constraints of the form
• With non-negative constraints

Slack Variables
• A mathematical representation of surplus resources.
• In real life problems, it’s unlikely that all resources will be used completely,
so there usually are unused resources.
• Slack variables represent the unused resources between the left-hand side
and right-hand side of each inequality.
x1 + 2x2 < 32
3x1 + 4x2 < 84
x1 + 2x2 + s1 = 32
3x1 + 4x2 + s2 = 84

Basic and Nonbasic Variables
• Basic variables are selected arbitrarily with the restriction that there be as
many basic variables as there are equations. The remaining variables are
non-basic variables.
• This system has two equations, we can select any two of the four variables
as basic variables. The remaining two variables are then non-basic
variables.
• A solution found by setting the two non-basic variables equal to 0 and
solving for the two basic variables is a basic solution. If a basic solution has
no negative values, it is a basic feasible solution.

Simple tableau
• Simplex Tableau: Most real-world problems are too complex to solve graphically.
They have too many corners to evaluate, and the algebraic solutions are lengthy. A
simplex tableau is a way to systematically evaluate variable mixes in order to find
the best one.
• Pivot Column: The column of the tableau representing the variable to be entered
into the solution mix.
• Pivot Row: The row of the tableau representing the variable to be replaced in the
solution mix.
• Pivot Number: The element in both the pivot column and the pivot row.
All Variables Solutions
Basic Variables Coefficients
0

Pivoting
• Pivoting :
– To transform a current tableau into the next one and increase the
value of the objective function to the next feasible solutions. To get to
that feasible solution several steps are performed and those steps are
called pivoting.
– Steps:
• First, divide all the entries of the pivot row by the pivot, its entry in the pivot column, to obtain
Rownew
• Then, replace each of the other rows, including the objective row, by the difference row − c
.Rownew
• where c is the row’s entry in the pivot column

Simplex Algorithm
• Diagram
• Steps
• Example
• Cycling & Bland’s rule
• Efficiency

SIMPLEX METHOD
Step-1
Write the
standard
maximization
problem in
standard form,
introduce slack
variables to form
the initial
system, and
write the initial
tableau.
Step-3
Select
the
pivot
column
Step-5
Select
the pivot
element
and
perform
the pivot
operatio
n
STOP
The optimal solution has been
found.
STOP
The linear programming problem
has no optimal solution
Step 2
Are there
any
negative
indicators
in the
bottom
row?
Step 4
Are there
any positive
elements in
the pivot
column
above the
dashed
line?
Simplex algorithm for standard maximization problems
YESYES
NONO

To Solve A Linear Programming Problem In Standard Form,
Use The Following Steps
• Convert each inequality in the set of constraints to an equation by adding slack
variables.
• Create the initial simplex tableau.
• Select the pivot column. ( The column with the “most negative value” element in
the last row.)
• Select the pivot row. (The row with the smallest non-negative result when the last
element in the row is divided by the corresponding in the pivot column.)
• Use elementary row operations calculate new values for the pivot row so that the
pivot is 1 (Divide every number in the row by the pivot number.)
• Use elementary row operations to make all numbers in the pivot column equal to
0 except for the pivot number. If all entries in the bottom row are zero or positive,
this the final tableau. If not, go back to step 3.
• If you obtain a final tableau, then the linear programming problem has a maximum
solution, which is given by the entry in the lower-right corner of the tableau.

Example(Step 1)

Example(Step 2)
u
v
z
1 1 1 0 4
1 3 0 1 6
-3 -5 0 0 0
Initialized by the coefficients of the
objective function. The signs are
reversed in this row (objective row)
We have constraints + 1
rows and variables +1
columns
Pivot Column/Entering Variable.
In our example, its Y

Example(Step 3)
x y u v
u 1 1 1 0 4
v 1 3 0 1 6
z -3 -5 0 0 0
Pivot Column/Entering Variable
The row with the smallest θ-ratio determines the
departing variable. For our example, it is variable v.
The intersecting element of Pivot Column and Pivot
Row is the pivot.
In the next tableau the intersected element will
become 1 and the rest of elements will become 0 in
the pivot column.
Departing Variable / Leaving Variable/ Pivot Row

Example(Step 4 Iteration 1)
x y u v
u 1 1 1 0 4
v 1 3 0 1 6
z -3 -5 0 0 0
x y u v
u 1 1 1 0 4
y 1/3 1 0 1/3 2
z -3 -5 0 0 0
x y u v
u 2/3 0 1 -1/3 2
y 1/3 1 0 1/3 2
z -4/3 0 0 5/3 10
The pivot row will be replaced by the Rownew we
calculated.
Row 1 and 3 will be calculated using the formula :
Row = Row – c.Rownew
Row1 –c.Rownew where c = 1
1-(1)(1/3), 1-(1)(1), 1-(1)(0), 0-(1)(1/3), 4-(1)(2)
2/3, 0, 1, -1/3, 2
Row3- c.Rownew where c = -5
-3-(-5)(1/3), -5-(-5)(1), 0-(-5)(0), 0-(-5)(1/3),
0-(-5)(2)
-4/3, 0, 0, 5/3, 10
All the values in the last row are nonnegative ??

Example(Step 4 Iteration 2)
x y u v
u 2/3 0 1 -1/3 2
y 1/3 1 0 1/3 2
z -4/3 0 0 5/3 10
x y u v
x 1 0 3/2 -1/2 3
y 0 1 -1/2 1/2 1
z 0 0 2 1 14
x y u v
x 1 0 3/2 -1/2 3
y 1/3 1 0 1/3 2
z -4/3 0 0 5/3 10
ϴ-Ratio: 3
ϴ-Ratio: 6
Row = Row – c.Rownew
Row2 –c.Rownew where c = 1/3
1/3-(1/3)(1), 1-(1/3)(0), 0-(1/3)(3/2),
1/3-(1/3)(-1/2), 2-(1/3)(3)
0, 1, -1/2, 1/2, 1
Row3- c.Rownew where c = -4/3
-4/3-(-4/3)(1), 0-(-4/3)(0), 0-(-4/3)(3/2),
5/3-(-4/3)(-1/2), 10-(-4/3)(3)
0, 0, 2, 1, 14
All the values in the last row are nonnegative ??
Rownew : 1, 0, 3/2, -1/2, 3

Example(Optimal Solution)
• x = 3
• y = 1
• u = 0
• v = 0
• z = 14
• We choose u and v equal to 0, because their column has all distinct values.
If they were like x and y, they’d have some value.

Cycling & Bland’s Rule
• When an objective function’s values “stall” for several iterations in a row
and that the algorithm cycles back to a previously considered point and
hence never terminate. This phenomenon is called cycling.
• Although, it rarely happens, but when it does ,a simple modification of the
simplex method, called Bland’s rule, eliminates even the theoretical
possibility of cycling.
• Assuming that the variables are denoted by a subscripted letter (e.g., x1,
x2,..., xn), this rule can be stated as follows:
– Among the columns with a negative entry in the objective row, select the
column with the smallest subscript in choosing pivot column.
– Resolve a tie among the smallest θ-ratios by selecting the row labeled by the
basic variable with the smallest subscript in choosing pivot row.

Software Simulation
• Simplex Method has became so efficient that its calculation
packages are even available online. One of them is as follows:
• For ease, we will copy the same problem to the online tool and
check whether it gives us the same result:
maximize z = 3x +5y
subject to :
x + y = 4
x+3y = 6
• Simplex Method Online Tool
http://www.zweigmedia.com/RealWorld/simplex.html

Efficiency of Simplex Algorithm
• Since we know that the algorithm progresses through a sequence of
adjacent points of a feasible region. Right?
• So, one should probably expect bad news because the number of extreme
points is known to grow exponentially with the problem size.
• The worst-case efficiency of the simplex method has been shown to be
exponential.
• But Fortunately!
• More than half a century of practical experience with the algorithm has
shown that the number of iterations in a typical application ranges
between m and 3m, with the number of operations per iteration
proportional to mn, where m and n are the numbers of equality
constraints and variables, respectively.

Other Variants of the Algorithm
• Ellipsoid Method:
– An important mile- stone in the history of such algorithms was the
proof by L. G. Khachian showing that the ellipsoid method can solve
any linear programming problem in polynomial time.
– However, ellipsoid method was much slower than the simplex method
in practice
• Karmarker’s Method
– Narendra Karmarkar published an algorithm that not only had a
polynomial worst-case efficiency but also was competitive with the
simplex method. This method generates a sequence of feasible
solutions that lie within the feasible region rather than going through
a sequence of adjacent extreme points as the simplex method does.
(Interior Point Methods).

Reference(s)
• The material presented in this lecture is adopted from:
• Introduction to Design and Analysis of Algorithms, 3rd Edition,
Anany Levitin.
• Introduction to Algorithms, Second Edition,
Thomas.H.Cormen
• www.iftikharahmad.org/teachings/Lec_16_Simplex_Algorith
m.pdf

Simplex Algorithm

More Related Content

What's hot (20)

Viewers also liked (11)

Similar to Simplex Algorithm (20)

Recently uploaded (20)

Simplex Algorithm