DynProg_Knapsack.ppt

Dynamic Programming …
Continued
0-1 Knapsack Problem

Last Class
 Longest Common Subsequence (LCS)
◦ Assigned problem – Find the LCS between
“HUMAN” and “CHIMPANZEE”
 Fill out the dynamic programming table
 Trace back what the LCS is
 Edit Distance
◦ Assigned problem – Find the edit distance
required to transform “KILT” into “KITTEN”
 Fill out the dynamic programming table
 Trace back the edit path
◦ Assigned problem – Outline the recursive calls
when using the recursive method for finding the
edit distance between “ON” and “OFF”.

Last Class
 Can I have volunteers to do the following on the
board?
1) Fill out the dynamic programming table for the LCS
between “HUMAN” and “CHIMPANZEE”
2) Trace back what the LCS string is between “HUMAN” and
“CHIMPANZEE”
3) Fill out the dynamic programming table to find the edit
distance required to transform “KILT” into “KITTEN”
4) Trace back the edit path that transformed “KILT” into
“KITTEN”
5) Outline the recursive calls when using the recursive
method for finding the edit distance between “ON” and
“OFF”.

Announcements
 Assignment #4 DNA Distance Problem

Knapsack 0-1 Problem
 The goal is to
maximize the value of
a knapsack that can
hold at most W units
(i.e. lbs or kg) worth of
goods from a list of
items I0, I1, … In-1.
◦ Each item has 2
attributes:
1) Value – let this be vi for
item Ii
2) Weight – let this be wi for
item Ii

 The difference
between this
problem and the
fractional knapsack
one is that you
CANNOT take a
fraction of an item.
◦ You can either take
it or not.
◦ Hence the name
Knapsack 0-1
problem.

 Brute Force
◦ The naïve way to solve this problem is to
cycle through all 2n subsets of the n items
and pick the subset with a legal weight
that maximizes the value of the knapsack.
◦ We can come up with a dynamic
programming algorithm that will USUALLY
do better than this brute force technique.

 As we did before we are going to solve the
problem in terms of sub-problems.
◦ So let’s try to do that…
 Our first attempt might be to characterize a
sub-problem as follows:
◦ Let Sk be the optimal subset of elements from {I0, I1, …, Ik}.
 What we find is that the optimal subset from the elements
{I0, I1, …, Ik+1} may not correspond to the optimal subset
of elements from {I0, I1, …, Ik} in any regular pattern.
◦ Basically, the solution to the optimization problem
for Sk+1 might NOT contain the optimal solution
from problem Sk.

 Let’s illustrate that point with an example:
Item Weight Value
I0 3 10
I1 8 4
I2 9 9
I3 8 11
 The maximum weight the knapsack can hold is 20.
 The best set of items from {I0, I1, I2} is {I0, I1, I2}
 BUT the best set of items from {I0, I1, I2, I3} is {I0,
I2, I3}.
◦ In this example, note that this optimal solution, {I0, I2, I3},
does NOT build upon the previous optimal solution, {I0, I1,
I2}.
 (Instead it build's upon the solution, {I0, I2}, which is really the

Knapsack 0-1 problem
 So now we must re-work the way we build upon previous
sub-problems…
◦ Let B[k, w] represent the maximum total value of a subset Sk with
weight w.
◦ Our goal is to find B[n, W], where n is the total number of items
and W is the maximal weight the knapsack can carry.
 So our recursive formula for subproblems:
B[k, w] = B[k - 1,w], if wk > w
= max { B[k - 1,w], B[k - 1,w - wk] + vk}, otherwise
 In English, this means that the best subset of Sk that has
total weight w is:
1) The best subset of Sk-1 that has total weight w, or
2) The best subset of Sk-1 that has total weight w-wk plus the item k

Knapsack 0-1 Problem –
Recursive Formula
 The best subset of Sk that has the total
weight w, either contains item k or not.
 First case: wk > w
◦ Item k can’t be part of the solution! If it was the
total weight would be > w, which is
unacceptable.
 Second case: wk ≤ w
◦ Then the item k can be in the solution, and we

Knapsack 0-1 Algorithm
for w = 0 to W { // Initialize 1st row to 0’s
B[0,w] = 0
}
for i = 1 to n { // Initialize 1st column to 0’s
B[i,0] = 0
}
for i = 1 to n {
for w = 0 to W {
if wi <= w { //item i can be in the solution
if vi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
}
else B[i,w] = B[i-1,w] // wi > w
}
}

 Let’s run our algorithm on the following
data:
◦ n = 4 (# of elements)
◦ W = 5 (max weight)
◦ Elements (weight, value):
(2,3), (3,4), (4,5), (5,6)

Knapsack 0-1 Example
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0
// Initialize the base cases
for w = 0 to W
B[0,w] = 0
for i = 1 to n
B[i,0] = 0

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0
if wi <= w //item i can be in the solution
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 1
vi = 3
wi = 2
w = 1
w-wi = -1
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0
2 0
3 0
4 0

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0
2 0
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 1
vi = 3
wi = 2
w = 2
w-wi = 0
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3
2 0
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3
2 0
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 1
vi = 3
wi = 2
w = 3
w-wi = 1
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3
2 0
3 0
4 0

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3
2 0
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 1
vi = 3
wi = 2
w = 4
w-wi = 2
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3
2 0
3 0
4 0

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3
2 0
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 1
vi = 3
wi = 2
w = 5
w-wi = 3
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0
3 0
4 0

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 2
vi = 4
wi = 3
w = 1
w-wi = -2
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 2
vi = 4
wi = 3
w = 2
w-wi = -1
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3
3 0
4 0

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 2
vi = 4
wi = 3
w = 3
w-wi = 0
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 2
vi = 4
wi = 3
w = 4
w-wi = 1
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4
3 0
4 0

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 2
vi = 4
wi = 3
w = 5
w-wi = 2
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0
4 0

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 3
vi = 5
wi = 4
w = 1..3
w-wi = -3..-1
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 3
vi = 5
wi = 4
w = 4
w-wi = 0
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 3
vi = 5
wi = 4
w = 5
w-wi = 1
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 4
vi = 6
wi = 5
w = 1..4
w-wi = -4..-1
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
i = 4
vi = 6
wi = 5
w = 5
w-wi = 0
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
B[i,w] = vi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]

Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
We’re DONE!!
The max possible value that can be carried in this knapsack is $7

 This algorithm only finds the max
possible value that can be carried in
the knapsack
◦ The value in B[n,W]
 To know the items that make this
maximum value, we need to trace
back through the table.

Finding the Items
 Let i = n and k = W
if B[i, k] ≠ B[i-1, k] then
mark the ith item as in the knapsack
i = i-1, k = k-wi
else
i = i-1 // Assume the ith item is not in the knapsack
// Could it be in the optimally packed knapsack?

Knapsack 0-1
Algorithm
Finding the Items
Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i = 4
k = 5
vi = 6
wi = 5
B[i,k] = 7
B[i-1,k] = 7
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
i = n , k = W
while i, k > 0
i = i-1, k = k-wi
else
i = i-1
Knapsac
k:

Knapsack 0-1
Algorithm
Finding the Items
Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i = 3
k = 5
vi = 5
wi = 4
B[i,k] = 7
B[i-1,k] = 7
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
i = n , k = W
while i, k > 0
i = i-1, k = k-wi
else
i = i-1
Knapsac
k:

Knapsack 0-1
Algorithm
Finding the Items
Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i = 2
k = 5
vi = 4
wi = 3
B[i,k] = 7
B[i-1,k] = 3
k – wi = 2
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
i = n , k = W
while i, k > 0
i = i-1, k = k-wi
else
i = i-1
Knapsac
k:
Item 2

Knapsack 0-1
Algorithm
Finding the Items
Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i = 1
k = 2
vi = 3
wi = 2
B[i,k] = 3
B[i-1,k] = 0
k – wi = 0
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
i = n , k = W
while i, k > 0
i = i-1, k = k-wi
else
i = i-1
Knapsac
k:
Item 1
Item 2

Knapsack 0-1
Algorithm
Finding the Items
Items:
1:
(2,3
)
2:
(3,4
)
3:
(4,5
)
4:
(5,6
)
i = 1
k = 2
vi = 3
wi = 2
B[i,k] = 3
B[i-1,k] = 0
k – wi = 0
i / w 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
k = 0, so we’re DONE!
The optimal knapsack should contain:
Item 1 and Item 2
Knapsac
k:
Item 1
Item 2

Knapsack 0-1 Problem – Run
Time
for w = 0 to W
B[0,w] = 0
for i = 1 to n
B[i,0] = 0
for i = 1 to n
for w = 0 to W
< the rest of the code >
What is the running time of this algorithm?
O(n*W)
Remember that the brute-force algorithm takes: O(2n)
O(W)
O(W)
Repeat n times
O(n)

Knapsack Problem
1) Fill out the
dynamic
programming
table for the
knapsack
problem to the
right.
2) Trace back
through the
table to find the
items in the

References
 Slides adapted from Arup Guha’s Computer
Science II Lecture notes:
http://www.cs.ucf.edu/~dmarino/ucf/cop350
3/lectures/
 Additional material from the textbook:
Data Structures and Algorithm Analysis in Java
(Second Edition) by Mark Allen Weiss
 Additional images:
www.wikipedia.com
xkcd.com

DynProg_Knapsack.ppt

More Related Content

Similar to DynProg_Knapsack.ppt (20)

More from Ruchika Sinha (20)

Recently uploaded (20)

DynProg_Knapsack.ppt