ai linMixed Integer Programming (MIP) is a type of mathematical optimization ear.ppt

Explorations in Artificial Intelligence
Prof. Carla P. Gomes
gomes@cs.cornell.edu
Module 7
Part 3
Integer Programming

Divisibility
Decision variables in an LP model are allowed to have any values,
including noninteger values, that satisfy the functional and
nonnegativity constraints. i.e., activities can be run at fractional levels.
What to do when divisibility assumption violated:
realm of integer programming!!!

Revisiting the TBAAirlines Problem
An Example where Integrality Matters

The TBA Airlines Problem
• TBA Airlines is a small regional company that specializes in short flights in small
airplanes.
• The company has been doing well and has decided to expand its operations.
• The basic issue facing management is whether to purchase more small airplanes to add
some new short flights, or start moving into the national market by purchasing some
large airplanes, or both.
Question: How many airplanes of each type should be purchased to maximize their
total net annual profit?

Data for the TBA Airlines Problem
Small
Airplane
Large
Airplane
Capital
Available
Net annual profit per airplane $1 million $5 million
Purchase cost per airplane 5 million 50 million $100 million
Maximum purchase quantity 2 —

Linear Programming Formulation
Let S = Number of small airplanes to purchase
L = Number of large airplanes to purchase
Maximize Profit = S + 5L ($millions)
subject to
Capital Available: 5S + 50L ≤ 100 ($millions)
Max Small Planes: S ≤ 2
and
S ≥ 0, L ≥ 0.

Graphical Method for Linear Programming
3
2
1
0 1 2 3 S
L
Feasible
region
Number
of large
airplanes
purchased
Number of small airplanes purchased
(2, 1) = Rounded solution
(Profit = 7)
(2, 1.8) = Optimal solution
Profit = 11 = S + 5 L

Violates Divisibility Assumption of LP
• Divisibility Assumption of Linear Programming: Decision variables in a linear
programming model are allowed to have any values, including fractional values, that
satisfy the functional and nonnegativity constraints. Thus, these variables are not
restricted to just integer values.
• Since the number of airplanes purchased by TBA must have an integer value, the
divisibility assumption is violated.

Integer Programming Formulation
Let S = Number of small airplanes to purchase
L = Number of large airplanes to purchase
Maximize Profit = S + 5L ($millions)
subject to
Capital Available: 5S + 50L ≤ 100 ($millions)
Max Small Planes: S ≤ 2
and
S ≥ 0, L ≥ 0
S, L are integers.

Graphical Method for Integer Programming
3
2
1
0 1 2 3 S
L
Number
of large
airplanes
purchased
Number of small airplanes purchased
(2, 1) = Rounded solution (Profit = 7)
(2, 1.8) = Optimal solution for the
LP relaxation (Profit = 11)
Profit = 10 = S + 5 L
(0, 2) = Optimal solution for the integer programming problem (Profit = 10)

Graphical Method for Integer Programming
• When an integer programming problem has just two decision variables, its optimal solution can be found
by applying the graphical method for linear programming with just one change at the end.
• We begin as usual by graphing the feasible region for the LP relaxation, determining the slope of the
objective function lines, and moving a straight edge with this slope through this feasible region in the
direction of improving values of the objective function.
• However, rather than stopping at the last instant the straight edge passes through this feasible region, we
now stop at the last instant the straight edge passes through an integer point that lies within this feasible
region.
• This integer point is the optimal solution.

Why integer programs?
• Advantages of restricting variables to take on integer values
– More realistic
– More flexibility
• Disadvantages
– More difficult to model
– Can be much more difficult to solve

Integer Programming
• When are “non-integer” solutions okay?
– Solution is naturally divisible
• e.g., $, pounds, hours
– Solution represents a rate
• e.g., units per week
– Solution only for planning purposes
• When is rounding okay?
– When numbers are large
• e.g., rounding 114.286 to 114 is probably okay.
• When is rounding not okay?
– When numbers are small
• e.g., rounding 2.6 to 2 or 3 may be a problem.
– Binary variables
• yes-or-no decisions

Types of Integer Programming Problems
• Pure integer programming problems are those where all the decision
variables must be integers.
• Mixed integer programming problems only require some of the variables
(the “integer variables”) to have integer values so the divisibility assumption
holds for the rest (the “continuous variables”).
• Binary variables are variables whose only possible values are 0 and 1.
• Binary integer programming (BIP) problems are those where all the
decision variables restricted to integer values are further restricted to be binary
variables.
– Such problems can be further characterized as either pure BIP problems or mixed
BIP problems, depending on whether all the decision variables or only some of
them are binary variables.

Examples of Applications of Binary Variables
• Making “yes-or-no” type decisions
– Build a factory?
– Manufacture a product?
– Do a project?
– Assign a person to a task?
• Logical constraints
– Alternative constraints
– Conditional constraints
• Representing non-linear functions
– Fixed Charge Problem
• If a product is produced, must incur a fixed setup cost.
• If a warehouse is operated, must incur a fixed cost.
– Piecewise linear representation
– Diseconomies of scale
– Approximation of nonlinear functions
• Set-covering, and set partitioning
– Make a set of assignments that “cover” a set of requirements.
– Partition a set into subsets meeting given requirements

StockCompany Example
Capital Budgeting Allocation Problem
StockCompany is considering 6 investments. The cash required from each
investment as well as the NPV of the investment is given next. The cash available
for the investments is $14,000. Stockco wants to maximize its NPV. What is the
optimal strategy?
An investment can be selected or not. One cannot select a fraction of an investment.

Data for the StockCompany Problem
Investment 1 2 3 4 5 6
Cash
Required
(1000s)
$5 $7 $4 $3 $4 $6
NPV
added
(1000s)
$16 $22 $12 $8 $11 $19
Investment budget = $14,000

Integer Programming Formulation
Max 16x1+ 22x2+ 12x3+ 8x4+ 11x5+ 19x6
5x1+ 7x2+ 4x3+ 3x4+ 4x5+ 6x6  14
1
0
, if we invest in i 1,...,6,
, else
i
x




What are the decision variables?
Objective and Constraints?

Capital Budgeting Allocation Problem (one resource)
Knapsack Problem
• Why is a problem with the characteristics of the previous problem called the
Knapsack Problem?
• It is an abstraction, considering the simple problem:
A hiker trying to fill her knapsack to maximum total value.
Each item she considers taking with her has a certain value and a
certain weight. An overall weight limitation gives the single
constraint.
Practical applications:
Project selection and capital budgeting allocation problems
Storing a warehouse to maximum value given the indivisibility of goods and space
limitations
Sub-problem of other problems e.g., generation of columns for a given model in the
course of optimization – cutting stock problem (beyond the scope of this course)

• The previous constraints represent “economic indivisibilities”, either a
project is selected, or it is not. There is no selecting of a fraction of a
project.
• Similarly, integer variables can model logical requirements (e.g., if
stock 2 is selected, then so is stock 1.)

How to model “logical” constraints
• Exactly 3 stocks are selected.
• If stock 2 is selected, then so is stock 1.
• If stock 1 is selected, then stock 3 is not selected.
• Either stock 4 is selected or stock 5 is selected, but not both.

Formulating Constraints
• Exactly 3 stocks are selected
x1+ x2+ x3+ x4+ x5+ x6=3

If stock 2 is selected then so is stock 1
 A 2-dimensional
representation
 Stock 2
 Stock 1
 The integer programming constraint:
x1  x2

If stock 1 is selected then stock 3 is not selected
 A 2-dimensional
representation
 Stock 3
 Stock 1
x1 + x3  1

Either stock 4 is selected or stock 5 is selected, but
not both.
 A 2-dimensional
representation
 stock 5
 stock 4
x4 + x5 = 1

California Manufacturing Company
• The California Manufacturing Company is a diversified company with several
factories and warehouses throughout California, but none yet in Los Angeles
or San Francisco.
• A basic issue is whether to build a new factory in Los Angeles or San
Francisco, or perhaps even both.
• Management is also considering building at most one new warehouse, but will
restrict the choice to a city where a new factory is being built.
Question: Should the California Manufacturing Company expand with
factories and/or warehouses in Los Angeles and/or San Francisco?

Data for California Manufacturing
Decision
Number
Yes-or-No
Question
Decision
Variable
Net Present
Value
(Millions)
Capital
Required
(Millions)
1 Build a factory in Los Angeles? x1 $8 $6
2 Build a factory in San Francisco? x2 5 3
3 Build a warehouse in Los Angeles? x3 6 5
4 Build a warehouse in San Francisco? x4 4 2
Capital Available: $10 million

Binary Decision Variables
Decision
Number
Decision
Variable
Possible
Value
Interpretation
of a Value of 1
Interpretation
of a Value of 0
1 x1 0 or 1
Build a factory in
Los Angeles
Do not build
this factory
2 x2 0 or 1
Build a factory in
San Francisco
Do not build
this factory
3 x3 0 or 1
Build a warehouse in
Los Angeles
Do not build
this warehouse
4 x4 0 or 1
Build a warehouse in
San Francisco
Do not build
this warehouse

Algebraic Formulation
Let x1 = 1 if build a factory in L.A.; 0 otherwise
x2 = 1 if build a factory in S.F.; 0 otherwise
x3 = 1 if build a warehouse in Los Angeles; 0 otherwise
x4 = 1 if build a warehouse in San Francisco; 0 otherwise
Maximize NPV = 8x1 + 5x2 + 6x3 + 4x4 ($millions)
subject to
Capital Spent: 6x1 + 3x2 + 5x3 + 2x4 ≤ 10 ($millions)
Max 1 Warehouse: x3 + x4 ≤ 1
Warehouse only if Factory: x3 ≤ x1
x4 ≤ x2
and
x1, x2, x3, x4 are binary variables.
Contingent decisions
Mutually exclusive decisions
Resource Availability

Mpl Model
Max 8x1 + 5x2 + 6x3 + 4x4;
subject to
6x1 + 3x2 + 5x3 + 2x4 <= 10;
x3 + x4 <= 1;
x3 <= x1;
x4 <= x2;
BINARY
x1;
x2;
x3;
x4;

Modeling Fixed Charge Problems
If a product is produced, must incur a fixed setup cost.
If a warehouse is operated, must incur a fixed cost.
The problem is non-linear.
x – quantity of product to be manufactured
x = 0  cost =0;
x > 0  cost = C1x + C2
How to model it? Using an indicator variable y
y = 1  x is produced; y = 0  x is not produced
Objective function becomes  C1x + C2y
Additional Constraint  x ≤ My

Wyndor with Setup Costs (Variation 1)
Suppose that two changes are made to the original Wyndor problem:
1. For each product, producing any units requires a substantial one-time setup
cost for setting up the production facilities.
2. The production runs for these products will be ended after one week, so D and
W in the original model now represent the total number of doors and windows
produced, respectively, rather than production rates. Therefore, these two
variables need to be restricted to integer values.

Graphical Solution to Original Wyndor Problem
0 2 4 6 8
8
6
4
2
Production rate
for windows
Production rate for doors
Feasible
Region
(2, 6)
Optimal solution
10
P = 3,600 = 300 D+ 500 W
W
D

Net Profit for Wyndor Problem with Setup Costs
Net Profit ($)
Number of
Units Produced Doors Windows
0 0(300) – 0 = 0 0 (500) – 0 = 0
1 1(300) – 700 = –400 1(500) – 1,300 = –800
2 2(300) – 700 = –100 2(500) – 1,300 = –300
3 3(300) – 700 = 200 3(500) – 1,300 = 200
4 4(300) – 700 = 500 4(500) – 1,300 = 700
5 Not feasible 5(500) – 1,300 = 1,200
6 Not feasible 6(500) – 1,300 = 1,700

Feasible Solutions for Wyndor with Setup Costs
0 2 4 6
2
4
6
8
Production
quantity for
windows
(0, 0)
gives P = 0
(4, 0) gives P = 500
(4, 3) gives P = 500 + 200
Production quantity for doors
(2, 6) gives P = -100 + 1700
(0, 6) gives P = 1700
8
= 1600
= 700
W
D
Optimal solution

Let D = Number of doors to produce,
W = Number of windows to produce,
y1 = 1 if perform setup to produce doors; 0 otherwise,
y2 = 1 if perform setup to produce windows; 0 otherwise .
Maximize P = 300D + 500W – 700y1 – 1,300y2
subject to
Original Constraints:
Plant 1: D ≤ 4
Plant 2: 2W ≤ 12
Plant 3: 3D + 2W ≤ 18
Produce only if Setup:
Doors: D ≤ My1
Windows: W ≤ My2
and
D ≥ 0, W ≥ 0, y1 and y2 are binary.

Wyndor with Mutually Exclusive Products
(Variation 2)
Suppose that now the only change from the original Wyndor problem is:
• The two potential new products (doors and windows) would compete for the
same customers. Therefore, management has decided not to produce both of
them together.
– At most one can be chosen for production, so either D = 0 or W = 0, or both.

Feasible Solution for
Wyndor with Mutually Exclusive Products
(for non-binary variables)
0 2 4 6
2
4
6
8
Production
rate for
windows
(0, 0) gives P = 0 (4, 0) gives P = 1,200
Production rate for doors
(0, 6) gives P = 3,000
P = 300 D + 500 W
Either D = 0 or W = 0
8
W
D

y1 = 1 if produce doors; 0 otherwise,
y2 = 1 if produce windows; 0 otherwise.
Maximize P = 300D + 500W
subject to
Original Constraints:
Plant 1: D ≤ 4
Plant 2: 2W ≤ 12
Plant 3: 3D + 2W ≤ 18
Auxiliary variables must =1 if produce any:
Doors: D ≤ My1
Windows: W ≤ My2
Mutually Exclusive: y1 + y2 ≤ 1
and
D ≥ 0, W ≥ 0, y1 and y2 are binary.

Wyndor with Either-Or Constraints
(Variation 3)
Suppose that now the only change from the original Wyndor problem is:
• The company has just opened a new plant (plant 4) that is similar to plant 3, so
the new plant can perform the same operations as plant 3 to help produce the
two new products (doors and windows).
• However, management wants just one of the plants to be chosen to work on
these new products. The plant chosen should be the one that provides the most
profitable product mix.

Data for Wyndor with Either-Or Constraints
(Variation 3)
Production Time Used for
Each Unit Produced (Hours)
Production Time
Available
per Week (Hours)
Plant Doors Windows
1 1 0 4
2 0 2 12
3 3 2 18
4 2 4 28
Unit Profit $300 $500

Graphical Solution with Plant 3 or Plant 4
0 2 4
2
4
6
8
0 2 4
2
4
6
8
(a) Choose Plant 3 (b) Choose Plant 4
(2, 6) gives
P = 3,600
(4, 5) gives
P = 3,700
Feasible
region
Feasible
region
W
W
D D

y = 1 if plant 4 is used; 0 if plant 3 is used
Maximize P = 300D + 500W
subject to
Plant 1: D ≤ 4
Plant 2: 2W ≤ 12
Plant 3: 3D + 2W ≤ 18 + My
Plant 4: 2D + 4W ≤ 28 + M(1 – y)
and
D ≥ 0, W ≥ 0, y is binary.

Applications of Binary Variables
• Making “yes-or-no” type decisions
– Build a factory?
– Manufacture a product?
– Do a project?
– Assign a person to a task?
• Fixed costs
– If a product is produced, must incur a fixed setup cost.
– If a warehouse is operated, must incur a fixed cost.
• Either-or constraints
– Production must either be 0 or ≥ 100.
• Subset of constraints
– meet 3 out of 4 constraints.

Special Kinds of Integer Programming Models
• Knapsack Problem
• Set Covering Problem
• Set Partitioning Problem
• Set Packing Problem
• The Traveling Salesman Problem
• The Quadratic Assignment Problem

Set Covering Problem
• We are given a set of objects S = {1, 2, 3, …, n}.
• We are also given a set of subsets of S, S. Each subset has
a cost associated with it.
• Problem:
– to “cover” all the members of S at the minimum cost using
members of S.
• Properties:
– The problem is a minimization and all constraints are >=;
– All RHS coefficients are 1;
– All other matrix coefficients are 0 or 1.

Fire Station Problem
1 2 3
4
5 6
7
8 9
11
10
12
14 15
13
16
Locate fire stations
so that each district
has a fire station in
it, or next to it.
Minimize the
number of fire
stations needed.

Representation as Set Covering Problem
1 2 3
4
5 6
7
8 9
11
10
12
14 15
13
16
Set Covers
1 1, 2, 4, 5
2 1, 2, 3, 5, 6
3 2, 3, 6, 7
16 13, 15, 16

Representation as Integer program
1 2 3
4
5 6
7
8 9
11
10
12
14 15
13
16
xj = 1 if node j is selected
xj = 0 otherwise
Minimize x1 + x2 + … + x16
s.t. x1 + x2 + x4 + x5  1
x1 + x2 + x3 + x5 + x6  1
x13 + x15 + x16  1
xj  {0, 1} for each j.
1
7
11
15

Southwestern Airways Crew Scheduling
• Southwestern Airways needs to assign crews to cover all its upcoming flights.
• We will focus on assigning 3 crews based in San Francisco (SFO) to 11
flights.
Question: How should the 3 crews be assigned 3 sequences of flights so that
every one of the 11 flights is covered?

Southwestern Airways Flights
Seattle
(SEA)
San Francisco
(SFO)
Los Angeles
(LAX)
Denver
(DEN)
Chicago
ORD)

Data for the Southwestern Airways Problem
Feasible Sequence of Flights (pairings)
Flights 1 2 3 4 5 6 7 8 9 10 11 12
1. SFO–LAX 1 1 1 1
2. SFO–DEN 1 1 1 1
3. SFO–SEA 1 1 1 1
4. LAX–ORD 2 2 3 2 3
5. LAX–SFO 2 3 5 5
6. ORD–DEN 3 3 4
7. ORD–SEA 3 3 3 3 4
8. DEN–SFO 2 4 4 5
9. DEN–ORD 2 2 2
10. SEA–SFO 2 4 4 5
11. SEA–LAX 2 2 4 4 2
Cost, $1,000s 2 3 4 6 7 5 7 8 9 9 8 9

Let xj = 1 if flight sequence (paring) j is assigned to a crew; 0 otherwise. (j = 1, 2, … , 12).
Minimize Cost = 2x1 + 3x2 + 4x3 + 6x4 + 7x5 + 5x6 + 7x7 + 8x8 + 9x9 + 9x10 + 8x11 + 9x12
(in $thousands)
subject to
Flight 1 covered: x1 + x4 + x7 + x10 ≥ 1
Flight 2 covered: x2 + x5 + x8 + x11 ≥ 1
: :
Flight 11 covered: x6 + x9 + x10 + x11 + x12 ≥ 1
Three Crews: x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10 + x11 + x12 ≤ 3
and
xj are binary (j = 1, 2, … , 12).
pairings

• We are given a set of objects S = {1, 2, 3, …, n}.
• We are also given S, a set of subsets of S. Each subset has
a cost associated with it.
• Problem:
– to “cover” all the members of S at the minimum cost using
members of S.
• Properties:
– The problem is a minimization and all constraints are >=;
– All RHS coefficients are 1;
– All other matrix coefficients are 0 or 1.

Some Comments on IP models
• There are often multiple ways of modeling the
same integer program.
• Solvers for integer programs are extremely
sensitive to the formulation. (not true for LPs)

Summary on Integer Programming
• Dramatically improves the modeling capability
– Economic indivisibilities
– Logical constraints
– Modeling nonlinearities (e.g., fixed cost)
– classical problems in capital budgeting and in
supply chain management
– Lots of other applications and models
• Not as easy to model
• Not as easy to solve.

The Challenges of Rounding
• Rounded Solution may not be
feasible.
• Rounded solution may not be
close to optimal.
• There can be many rounded
solutions.
– Example: Consider a problem
with 30 variables that are non-
integer in the LP-solution.
How many possible rounded
solutions are there?
1 2 3 4 5
1
2
3
4
5
x1
x2

Overview of Techniques for Solving Integer
Programs
• Enumeration Techniques
– Complete Enumeration
• list all “solutions” and choose the best
– Branch and Bound
• Implicitly search all solutions, but cleverly eliminate the vast majority before
they are even searched
– Implicit Enumeration
• Branch and Bound applied to binary variables
• Cutting Plane Techniques
– Use LP to solve integer programs by adding constraints to eliminate the fractional
solutions.

How Integer Programs are Solved
1 2 3 4 5
1
2
3
4
5
x1
x2

Branch and Bound
– It is the starting point for all solution techniques for
integer programming
– Lots of research has been carried out over the past 40
years to make it more and more efficient
– But, it is an art form to make it efficient. (We shall get
a sense why.)
– Integer programming is intrinsically difficult.

Investment 1 2 3 4 5 6
Cash
Required
(1000s)
$5 $7 $4 $3 $4 $6
NPV
added
(1000s)
$16 $22 $12 $8 $11 $19
Capital Budgeting Example
Investment budget = $14,000
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6  14
x binary for j = 1 to 6

Complete Enumeration
• Systematically considers all possible values of the decision
variables.
– If there are n binary variables, there are 2n
different
ways.
• Usual idea: iteratively break the problem in two. At the
first iteration, we consider separately the case that x1 = 0
and x1 = 1.

An Enumeration Tree
x1 = 0 x1 = 1
x2 = 0 x2 = 1 x2 = 0 x2 = 1
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
Original
problem

On complete enumeration
• Suppose that we could evaluate 1 billion solutions per
second.
• Let n = number of binary variables
• Solutions times (approx.)
– n = 30, 1 second
– n = 40, 18 minutes
– n = 50 13 days
– n = 60 31 years

On complete enumeration
• Suppose that we could evaluate 1 trillion solutions per
second, and instantaneously eliminate 99.9999999% of all
solutions as not worth considering
• Let n = number of binary variables
• Solutions times
– n = 70, 1 second
– n = 80, 17 minutes
– n = 90 11.6 days
– n = 100 31 years

Branch and Bound
The essential idea: search the enumeration tree, but at
each node
1. Solve the linear program at the node
2. Eliminate the subtree (fathom it) if
1. The solution is integer (there is no need to go further-
why?) or
2. The best solution in the subtree cannot be as good as
the best available solution (the incumbent- how does
that happen?) or
3. There is no feasible solution

Branch and Bound
1 44 3/7
Solution at node 1:
x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7
Node 1 is the
original LP
Relaxation
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
0  xj  1 for j = 1 to 6
The IP cannot have value higher than 44 3/7.

Branch and Bound
1
2
x1 = 0
44 3/7
44
Solution at node 2:
x1 = 0 x2 = 1 x3 = 1/4 x4 = x5 = 0 x6 = 1 z = 44
Node 2 is the
original LP
Relaxation plus
the constraint
x1 = 0.
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
0  xj  1 for j = 1 to 6, x1 = 0

Branch and Bound
1
2
x1 = 0
44 3/7
44
Node 3 is the
original LP
Relaxation plus
the constraint
x1 = 1.
3
x1 = 1
The solution at node 1 was
x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7
Note: it was the best solution with no constraint on x1.
So, it is also the solution for node 3. (If you add a
constraint, and the old optimal solution is feasible, then
it is still optimal.)
44 3/7

Branch and Bound
1
2 3
x1 = 0 x1 = 1
44 3/7
44 44 3/7
Solution at node 4: 0 0 1 0 1 1 z = 42
4
x2 = 0
42
Our first incumbent solution!
No further searching from node 4 because
there cannot be a better integer solution.
Node 4 is the
original LP
Relaxation plus
the constraints
x1 = 0, x2 = 0.
4
No solution in
the subtree can
have a value
better than 42.
1

Branch and Bound
1
2 3
x1 = 0 x1 = 1
44 3/7
44 44 3/7
We next solved the LP’s associated with nodes 5, 6, and 7
4
x2 = 0
42
No new integer solutions were found.
5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44 1/3
The
incumbent
solution has
value 42
4
We would eliminate (fatham) a subtree if we were guaranteed
that no solution in the subtree were better than the incumbent.
The incumbent
is the best
solution on
hand.
1

Branch and Bound
1
2 3
x1 = 0 x1 = 1
44 3/7
44 44 3/7
We next solved the LP’s associated with nodes 8 -13
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44 1/3
The
incumbent
solution has
value 42
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43.75 43.5 43.25 43.8 44.3 -
4
13

Summary so far
• We have solved 13 different linear programs so far.
– One integer solution found
– One subtree fathomed (pruned) because the solution
was integer (node 4)
– One subtree fathomed because the solution was
infeasible (node 13)
– No subtrees fathomed because of the bound

Branch and Bound
1
2 3
x1 = 0 x1 = 1
44 3/7
44 44 3/7
We next solved the LP’s associated with the next nodes.
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44 1/3
The
incumbent
solution has
value 42
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43.75 43.5 43.25 43.8 44.3 -
43.75 42.66
We can fathom the node with z = 42.66. Why?

Getting a better bound
• The bound at each node is obtained by solving an LP.
• But all costs are integer, and so the objective value of each
integer solution is integer. So, the best integer solution has an
integer objective value.
• If the best integer valued solution for a node is at most 42.66,
then we know the best bound is at most 42.
• Other bounds can also be rounded down.

Branch and Bound
1
2 3
x1 = 0 x1 = 1
44 3/7
44 44 3/7
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44 1/3
The
incumbent
solution has
value 42
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43.75 43.5 43.25 43.8 44.3 -.
43.75 42.66

Branch and Bound
1
2 3
x1 = 0 x1 = 1
44
44 44
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44
The
incumbent
solution has
value 42
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43 43 43 43 44 -
43 42 43 42 43 43
We found a new incumbent solution!
x1 = 1, x2 = x3 = 0, x4 = 1, x5 = 0, x6 = 1 z = 43

Branch and Bound
1
2 3
x1 = 0 x1 = 1
44
44 44
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44
The new
incumbent
solution has
value 43
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43 43 43 43 44 -
43 42 43 42 43 43
We found a new incumbent solution!
x1 = 1, x2 = x3 = 0, x4 = 1, x5 = 0, x6 = 1 z = 43
8 9 10 11

Branch and Bound
1
2 3
x1 = 0 x1 = 1
44
44 44
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44
The new
incumbent
solution has
value 43
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43 43 43 43 44 -
If we had found this incumbent earlier, we could have
saved some searching.
8 9 10 11

Finishing Up
1
2 3
x1 = 0 x1 = 1
44
44 44
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44
The new
incumbent
solution has
value 43
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43 43 43 43 44 -
8 9 10 11
-
-
14 15
16 17
44
44
18 19 -
38

Lessons Learned
• Branch and Bound can speed up the search
• Only 25 nodes (linear programs) were evaluated
• Other nodes were fathomed
• Obtaining a good incumbent earlier can be valuable
• only 19 nodes would have been evaluated.
• Solve linear programs faster, because we start with an excellent or optimal solution
• uses a technique called the dual simplex method
• Obtaining better bounds can be valuable.
• We sometimes use properties that are obvious to us, such as the fact that integer solutions have
integer solution values

Branch and Bound
Notation:
– z* = optimal integer solution value
– Subdivision: a node of the B&B Tree
– Incumbent: the best solution on hand
– zI
: value of the incumbent
– zLP
: value of the LP relaxation of the current node
– Children of a node: the two problems created for a
node, e.g., by saying xj = 1 or xj = 0.
– LIST: the collection of active (not fathomed) nodes,
with no active children.
NOTE: zI
 z*

Illustrating the definitions
1
2 3
x1 = 0 x1 = 1
44 3/7
44 44 3/7
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44 1/3
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43.75 43.5 43.25 43.8 44.3 -
4
13
z* = 43 = optimal integer solution value. (We
found it later in the search)
Incumbent is 0 0 1 0 1 1 zI
= 42.
It is the optimal solution for the subdivision 4.
The incumbent
solution has
value 42
The zLP
values for each subdivision are next to the nodes.

Illustrating the definitions
1
2 3
x1 = 0 x1 = 1
44 3/7
44 44 3/7
4
x2 = 0
42 5
x2 = 1
6
x2 = 0
7
x2 = 1
44 44 44 1/3
8
x3 = 0
9
x3 = 1
10
x3 = 0
11
x3 = 1
12
x3 = 0
13
x3 = 1
43.75 43.5 43.25 43.8 44.3 -
4
13
The children of node (subdivision) 1 are nodes 2 and 3.
The children of node 3 are nodes 6 and 7.
LIST = { 8, 9, 10, 11, 12 } = unfathomed nodes with no
active children
The incumbent
solution has
value 42

Branch and Bound Algorithm
INITIALIZE LIST = {original problem}
Incumbent: = 
zI
= -
SELECT:
If LIST = , then the Incumbent is optimal if it exists, and
the problem is infeasible if no incumbent exists;
else, let S be a node (subdivision) from LIST.
Let xLP
be the optimal solution to S
Let zLP
= its objective value
CASE 1. zLP
= - (the LP is infeasible)
Remove S from LIST (fathom it)
Return to SELECT
1
44 3/7
e.g., S = {1}
13
-
e.g., S = {13} 13

INITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and
else, let S be a node from LIST.
Let xLP
Let zLP
CASE 2. - < zLP
 zI
.
That is, the LP is dominated by the incumbent.
Then remove S from LIST (fathom it)
Return to SELECT
8
43
14
43
8
42
e.g., the incumbent has value 43, and node
14 is selected. zLP
= 43.
14

INITIALIZE
SELECT:
else, let S be a subdivision from LIST.
Let xLP
Let zLP
CASE 3. zI
< zLP
and xLP
is integral.
That is, the LP solution is integral and
dominates the incumbent.
Then Incumbent := xLP
;
zI
:= zLP
Remove S from LIST (fathomed by integrality)
Return to SELECT
42 4
4
e.g., node 4 was selected,
and the solution to the
LP was integer-valued.

INITIALIZE
SELECT:
else, let S be a subdivision from LIST.
Let xLP
Let zLP
CASE 4. zI
< zLP
and xLP
is not integral.
There is not enough information to fathom S
Remove S from LIST
Add the children of S to LIST
Return to SELECT
44 3/7
6
x2 = 0
7
x2 = 1
3
x1 = 1
e.g., select node 3.
List := List – 3 + {6,7}

Different Selection Rules are Possible
• Rule of Thumb 1: Don’t let LIST get too big (the solutions
must be stored). So, prefer nodes that are further down in
the tree.
• Rule of Thumb 2: Pick a node of LIST that is likely to lead
to an improved incumbent. Sometimes special heuristics
are used to come up with a good incumbent.

X X
X X
X
X
X
X
X
X
X
2
1 1
2
3
3 3
3 3
3
3
= 0
= 0
= 1
= 0 = 1
= 0 = 1
= 0
= 0 = 1
= 1 X X8
8 = 1
= 0
= 1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Branching
One does not have to have the B&B
tree be symmetric, and one does not
select subtrees by considering variables
in order.
Choosing how to
branch so as to
reduce running
time is largely
“art” and based
on experience.
X3

Different Branching Rules are Possible
• Branching: determining children for a node. There are many choices.
• Rule of thumb 1: if it appears clear that xj = 1 in an optimal solution,
it is often good to branch on xj = 0 vs xj = 1.
– The hope is that a subdivision with xj = 0 can be pruned.
• Rule of thumb 2: branching on important variables is worthwhile

Different Bounding Techniques are Possible
• We use the bound obtained by dropping the integrality
constraints (LP relaxation). There are other choices.
• Key tradeoff for bounds: time to obtain a bound vs quality
of the bound.
• If one can obtain a bound much quicker, sometimes we
would be willing to get a bound that is worse
• It usually is worthwhile to get a bound that is better, so
long as it doesn’t take too long.

What if the variables are general integer variables?
• One can choose children as follows:
– child 1: x1  3 (or xj  k)
– child 2 x1  4 (or xj  k+1)
• How would one choose the variable j and the value k
– A common choice would be to take a fractional value
from xLP
. e.g., if x7 = 5.62, then we may branch on x7 
5 and x7  6.
– Other choices are also possible.

• Branch and Bound is the standard way of solving IPs to optimality.
• There is art to making it work well in practice.
• Much of the art is built into state-of-the-art solvers such as CPLEX.
New exciting area
combining LP based branch-and-bound based
techniques with constraint programming
techniques - forward checking; arc-consistency
and other forms of consistency checking and
propagation!

A* - well-known AI algorithm that is a
generalization of branch-and-bound with
LP relaxations – notion of admissible
heuristic that overestimates
(underestimates) the objective function for
a maximization (minimization) problem
(analogously to what a relaxation does)
Used in e.g. MapQuest and Darpa Challenge

It was fun to teach INFO 372!
Hope you had fun too!
THE END
!!!

ai linMixed Integer Programming (MIP) is a type of mathematical optimization ear.ppt

More Related Content

Similar to ai linMixed Integer Programming (MIP) is a type of mathematical optimization ear.ppt (20)

Recently uploaded (20)

ai linMixed Integer Programming (MIP) is a type of mathematical optimization ear.ppt

Editor's Notes