dynamic-programming unit 3 power point presentation

7 -2
Dynamic Programming
 Dynamic Programming is an
algorithm design method that can be
used when the solution to a problem
may be viewed as the result of a
sequence of decisions

3
Coin set for examples
 For the following examples, we will assume coins in the
following denominations:
1¢ 5¢ 10¢ 21¢ 25¢
 We’ll use 63¢ as our goal

4
A dynamic programming solution
 Idea: Solve first for one cent, then two cents, then three cents,
etc., up to the desired amount
 Save each answer in an array !
 For each new amount N, compute all the possible pairs of
previous answers which sum to N
 For example, to find the solution for 13¢,

First, solve for all of 1¢, 2¢, 3¢, ..., 12¢

Next, choose the best solution among:
 Solution for 1¢ + solution for 12¢

5
Example
 Suppose coins are 1¢, 3¢, and 4¢
 There’s only one way to make 1¢ (one coin)
 To make 2¢, try 1¢+1¢ (one coin + one coin = 2 coins)
 To make 3¢, just use the 3¢ coin (one coin)
 To make 4¢, just use the 4¢ coin (one coin)
 To make 5¢, try

1¢ + 4¢ (1 coin + 1 coin = 2 coins)

2¢ + 3¢ (2 coins + 1 coin = 3 coins)

The first solution is better, so best solution is 2 coins
 To make 6¢, try

1¢ + 5¢ (1 coin + 2 coins = 3 coins)

2¢ + 4¢ (2 coins + 1 coin = 3 coins)

3¢ + 3¢ (1 coin + 1 coin = 2 coins) – best solution
 Etc.

6
How good is the algorithm?
 When algorithm is recursive, with a branching factor of
up to 62
 Possibly the average branching factor is somewhere around
half of that (31)
 The algorithm takes exponential time, with a large base
 The dynamic programming algorithm is O(N*K), where
N is the desired amount and K is the number of different
kinds of coins

7
Comparison with divide-and-conquer
 Divide-and-conquer algorithms split a problem into separate
subproblems, solve the subproblems, and combine the results for
a solution to the original problem
 Example: Quicksort
 Example: Mergesort
 Example: Binary search
 Divide-and-conquer algorithms can be thought of as top-down
algorithms
 In contrast, a dynamic programming algorithm proceeds by
solving small problems, then combining them to find the solution
to larger problems
 Dynamic programming can be thought of as bottom-up

Dynamic Programming
Dynamic Programming is a general algorithm design technique
for solving problems defined by or formulated as recurrences with
overlapping subinstances
• Invented by American mathematician Richard Bellman in the 1950s
to solve optimization problems and later assimilated by CS
• Main idea:
- set up a recurrence relating a solution to a larger instance to
solutions of some smaller instances
- solve smaller instances once
- record solutions in a table
- extract solution to the initial instance from that table

Example: Fibonacci numbers
• Recall definition of Fibonacci numbers:
F(n) = F(n-1) + F(n-2)
F(0) = 0
F(1) = 1
• Computing the nth
Fibonacci number recursively (top-down):
F(n)
F(n-1) + F(n-2)
F(n-2) + F(n-3) F(n-3) + F(n-4)
...

Example: Fibonacci numbers (cont.)
Computing the nth
Fibonacci number using bottom-up iteration and
recording results:
F(0) = 0
F(1) = 1
F(2) = 1+0 = 1
…
F(n-2) =
F(n-1) =
F(n) = F(n-1) + F(n-2)
Efficiency:
- time
- space
0 1 1 . . . F(n-2) F(n-1) F(n)
n
n

Computing a binomial coefficient by DP
Binomial coefficients are coefficients of the binomial formula:
(a + b)n
= C(n,0)an
b0
+ . . . + C(n,k)an-k
bk
+ . . . + C(n,n)a0
bn
Recurrence: C(n,k) = C(n-1,k) + C(n-1,k-1) for n > k > 0
C(n,0) = 1, C(n,n) = 1 for n  0
Value of C(n,k) can be computed by filling a table:
0 1 2 . . . k-1 k
0 1
1 1 1
.
.
.
n-1 C(n-1,k-1) C(n-1,k)
n C(n,k)

Computing C(n,k): pseudocode and analysis
Time efficiency: Θ(nk)
Space efficiency: Θ(nk)

13
Solution by dynamic programming
 n c(n,0) c(n,1) c(n,2) c(n,3) c(n,4) c(n,5) c(n,6)
 0 1
 1 1 1
 2 1 2 1
 3 1 3 3 1
 4 1 4 6 4 1
 5 1 5 10 10 5 1
 6 1 6 15 20 15 6 1
 Each row depends only on the preceding row
 Only linear space and quadratic time are needed
 This algorithm is known as Pascal’s Triangle

14
The principle of optimality, I
 Dynamic programming is a technique for finding an
optimal solution
 The principle of optimality applies if the optimal
solution to a problem always contains optimal solutions
to all subproblems
 Example: Consider the problem of making N¢ with the
fewest number of coins
 Either there is an N¢ coin, or
 The set of coins making up an optimal solution for N¢ can be
divided into two nonempty subsets, n1¢ and n2¢

If either subset, n1¢ or n2¢, can be made with fewer coins, then clearly
N¢ can be made with fewer coins, hence solution was not optimal

15
The principle of optimality, II
 The principle of optimality holds if
 Every optimal solution to a problem contains...
 ...optimal solutions to all subproblems
 The principle of optimality does not say
 If you have optimal solutions to all subproblems...
 ...then you can combine them to get an optimal solution
 Example: In US coinage,
 The optimal solution to 7¢ is 5¢ + 1¢ + 1¢, and
 The optimal solution to 6¢ is 5¢ + 1¢, but
 The optimal solution to 13¢ is not 5¢ + 1¢ + 1¢ + 5¢ + 1¢
 But there is some way of dividing up 13¢ into subsets with
optimal solutions (say, 11¢ + 2¢) that will give an optimal
solution for 13¢
 Hence, the principle of optimality holds for this problem

16
Longest simple path
 Consider the following
graph:
 The longest simple path (path not containing a cycle) from A
to D is A B C D
 However, the subpath A B is not the longest simple path
from A to B (A C B is longer)
 The principle of optimality is not satisfied for this problem
 Hence, the longest simple path problem cannot be solved by
a dynamic programming approach
A C D
B
4
2
3
1
1

Multistage Graphs
 A multistage graph G = (V,E) is a directed graph in
which vertices are partitioned into k ≥ 2 disjoint sets Vi,
1 ≤ i ≤ k.
 (u,v) is an edge in E, then u belongs to Vi and v belongs
to Vi+1 for some i, 1 ≤ i ≤ k.
 The sets V1 and Vk are such that |V1| = |Vk| = 1.
 The multistage graph problem is to find a minimum cost
path from s (source at V1) to t(sink at Vk).
 Because of the constraints on E, every path from s to t
starts in stage 1, goes to stage 2, and so on and
eventually terminates in stage k.

The shortest path
 To find a shortest path in a multi-stage graph
 Apply the greedy method :
the shortest path from S to T :
1 + 2 + 5 = 8
S A B T
3
4
5
2 7
1
5 6

The shortest path in multistage graphs
 e.g.
 The greedy method can not be applied to this case:
(S, A, D, T) 1+4+18 = 23.
 The real shortest path is:
(S, C, F, T) 5+2+2 = 9.
S T
13
2
B E
9
A D
4
C F
2
1
5
11
5
16
18
2

Dynamic programming approach
 Dynamic programming approach (forward approach):
 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
S T
2
B
A
C
1
5
d(C, T)
d(B, T)
d(A, T)
A
T
4
E
D
11
d(E, T)
d(D, T)
 d(A,T) = min{4+d(D,T), 11+d(E,T)}
= min{4+18, 11+13} = 22.
S T
13
2
B E
9
A D
4
C F
2
1
5
11
5
16
18
2

 d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}
= min{9+18, 5+13, 16+2} = 18.
 d(C, T) = min{ 2+d(F, T) } = 2+2 = 4
 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
 The above way of reasoning is called backward reasoning.
S T
13
2
B E
9
A D
4
C F
2
1
5
11
5
16
18
2

Backward approach (forward reasoning)
 d(S, A) = 1
d(S, B) = 2
d(S, C) = 5
 d(S,D)=min{d(S,A)+d(A,D), d(S,B)+d(B,D)}
= min{ 1+4, 2+9 } = 5
d(S,E)=min{d(S,A)+d(A,E), d(S,B)+d(B,E)}
= min{ 1+11, 2+5 } = 7
d(S,F)=min{d(S,B)+d(B,F), d(S,C)+d(C,F)}
= min{ 2+16, 5+2 } = 7
S T
13
2
B E
9
A D
4
C F
2
1
5
11
5
16
18
2

 d(S,T) = min{d(S, D)+d(D, T), d(S,E)+
d(E,T), d(S, F)+d(F, T)}
= min{ 5+18, 7+13, 7+2 }
= 9
S T
13
2
B E
9
A D
4
C F
2
1
5
11
5
16
18
2

Principle of optimality
 Principle of optimality: Suppose that in solving a
problem, we have to make a sequence of decisions
D1
, D2
, …, Dn
. If this sequence is optimal, then the
last k decisions, 1  k  n must be optimal.
 e.g. the shortest path problem
If i, i1
, i2
, …, j is a shortest path from i to j, then i1
, i2
,
…, j must be a shortest path from i1
to j
 In summary, if a problem can be described by a
multistage graph, then it can be solved by dynamic
programming.

 Forward approach and backward approach:
 Note that if the recurrence relations are formulated
using the forward approach then the relations are
solved backwards . i.e., beginning with the last decision
 On the other hand if the relations are formulated using
the backward approach, they are solved forwards.
 To solve a problem by using dynamic
programming:
 Find out the recurrence relations.
 Represent the problem by a multistage graph.
Dynamic programming

26
The 0-1 knapsack problem
 A thief breaks into a house, carrying a knapsack...
 He can carry up to 25 pounds of loot
 He has to choose which of N items to steal

Each item has some weight and some value

“0-1” because each item is stolen (1) or not stolen (0)
 He has to select the items to steal in order to maximize the value of his
loot, but cannot exceed 25 pounds
 A greedy algorithm does not find an optimal solution
 A dynamic programming algorithm works well
 This is similar to, but not identical to, the coins problem
 In the coins problem, we had to make an exact amount of change
 In the 0-1 knapsack problem, we can’t exceed the weight limit, but the
optimal solution may be less than the weight limit
 The dynamic programming solution is similar to that of the coins problem

27
Comments
 Dynamic programming relies on working “from the bottom up”
and saving the results of solving simpler problems
 These solutions to simpler problems are then used to compute the solution
to more complex problems
 Dynamic programming solutions can often be quite complex and
tricky
 Dynamic programming is used for optimization problems,
especially ones that would otherwise take exponential time
 Only problems that satisfy the principle of optimality are suitable for
dynamic programming solutions
 Since exponential time is unacceptable for all but the smallest
problems, dynamic programming is sometimes essential

Algorithm Fgraph(G,k,n,p)
{
cost[n]=0.0;
for j:=n-1 to 1 step -1 do
{
Let r be a vertex such that <j,r> is an edge of G and
c[j,r}+cost[r] is minimum;
cost[j]:=c[j,r]+cost[r];
d[j]:=r;
}
p[1]:=1;
p[k]:=n;
for j:=2 to k-1 do
p[j]:=d[p[j-1]];
}

Algorithm Bgraph(G,k,n,p)
{
bcost[1]=0.0;
for j:=2 to n do
{
Let r be a vertex such that <r,j> is an edge of G and c[r,j]+bcost[r] is
minimum;
bcost[j]:=c[r,j]+bcost[r];
d[j]:=r;
}
p[1]:=1;
p[k]:=n;
for j:=k-1 to 2 step -1 do
p[j]:=d[p[j+1]];
}

Time Complexity:

The time taken by this algorithm is Θ (|v|+|E|) , if
adjacency lists are used for representing the graph.

Then r can be found in time proportional to degree
of vertex j.

Hence G has |E| edges the time for outer for loop is
Θ(|v|+|E|)

dynamic-programming unit 3 power point presentation

More Related Content

Similar to dynamic-programming unit 3 power point presentation (20)

More from Shrinivasa6 (12)

Recently uploaded (20)

dynamic-programming unit 3 power point presentation