ch09-04-14-14.ppt design and analysis of algorithms
- 1. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 1 Greedy Technique Greedy Technique Constructs a solution to an Constructs a solution to an optimization problem optimization problem piece by piece by piece through a sequence of choices that are: piece through a sequence of choices that are: feasible feasible locally optimal locally optimal irrevocable irrevocable For some problems, yields an optimal solution for every instance. For some problems, yields an optimal solution for every instance. For most, does not but can be useful for fast approximations. For most, does not but can be useful for fast approximations.
- 2. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 2 Applications of the Greedy Strategy Applications of the Greedy Strategy Optimal solutions: Optimal solutions: • change making for “normal” coin denominations change making for “normal” coin denominations • minimum spanning tree (MST) minimum spanning tree (MST) • single-source shortest paths single-source shortest paths • simple scheduling problems simple scheduling problems • Huffman codes Huffman codes Approximations: Approximations: • traveling salesman problem (TSP) traveling salesman problem (TSP) • knapsack problem knapsack problem • other combinatorial optimization problems other combinatorial optimization problems
- 3. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 3 Change-Making Problem Change-Making Problem Given unlimited amounts of coins of denominations Given unlimited amounts of coins of denominations d d1 1 > … > > … > d dm m , , give change for amount give change for amount n n with the least number of coins with the least number of coins Example: Example: d d1 1 = 25c, = 25c, d d2 2 =10c, =10c, d d3 3 = 5c, = 5c, d d4 4 = 1c and = 1c and n = n = 48c 48c Greedy solution: Greedy solution: Greedy solution: Greedy solution: optimal for any amount and “typical’’ set of denominations optimal for any amount and “typical’’ set of denominations not optimal for all coin denominations … not optimal for all coin denominations …
- 4. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 4 Change-Making Problem Change-Making Problem Greedy not optimal for all sets of denominations: Greedy not optimal for all sets of denominations: Consider 1, 3, 4 Consider 1, 3, 4 For what value does greedy algorithm fail? For what value does greedy algorithm fail?
- 5. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 5 Minimum Spanning Tree (MST) Minimum Spanning Tree (MST) Spanning tree Spanning tree of a connected graph of a connected graph G G: a connected acyclic : a connected acyclic subgraph of subgraph of G G that includes all of that includes all of G G’s vertices ’s vertices Minimum spanning tree Minimum spanning tree of a weighted, connected graph of a weighted, connected graph G G: : a spanning tree of a spanning tree of G G of minimum total weight of minimum total weight Example: Example: c d b a 6 2 4 3 1
- 6. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 6 Prim’s MST algorithm Prim’s MST algorithm Start with tree Start with tree T T1 1 consisting of one (any) vertex and “grow” consisting of one (any) vertex and “grow” tree one vertex at a time to produce MST through tree one vertex at a time to produce MST through a series of a series of expanding subtrees T expanding subtrees T1 1, T , T2 2, …, T , …, Tn n On each iteration, On each iteration, construct T construct Ti i+1 +1 from T from Ti i by adding vertex by adding vertex not in not in T Ti i that is that is closest to those already in closest to those already in T Ti i (this is a (this is a “greedy” step!) “greedy” step!) Stop when all vertices are included Stop when all vertices are included
- 7. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 7 Example Example c d b a 4 2 6 1 3
- 8. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 8 Notes about Prim’s algorithm Notes about Prim’s algorithm Proof by induction that this construction actually yields MST Proof by induction that this construction actually yields MST Needs priority queue for locating closest fringe vertex Needs priority queue for locating closest fringe vertex Efficiency Efficiency • O( O(n n2 2 ) ) for weight matrix representation of graph and array for weight matrix representation of graph and array implementation of priority queue implementation of priority queue • O O( (m m log log n n) for adjacency list representation of graph with ) for adjacency list representation of graph with n n vertices and vertices and m m edges and min-heap implementation of edges and min-heap implementation of priority queue priority queue
- 9. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 9 Another greedy algorithm for MST: Kruskal’s Another greedy algorithm for MST: Kruskal’s Sort the edges in nondecreasing order of lengths Sort the edges in nondecreasing order of lengths “ “Grow” tree one edge at a time to produce MST through Grow” tree one edge at a time to produce MST through a a series of expanding forests F series of expanding forests F1 1, F , F2 2, …, F , …, Fn- n-1 1 On each iteration, add the next edge on the sorted list On each iteration, add the next edge on the sorted list unless this would create a cycle. (If it would, skip the edge.) unless this would create a cycle. (If it would, skip the edge.)
- 10. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 10 Example Example c d b a 4 2 6 1 3
- 11. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 11 Notes about Kruskal’s algorithm Notes about Kruskal’s algorithm Algorithm looks easier than Prim’s but is harder to Algorithm looks easier than Prim’s but is harder to implement (checking for cycles!) implement (checking for cycles!) Cycle checking: a cycle is created iff added edge connects Cycle checking: a cycle is created iff added edge connects vertices in the same connected component vertices in the same connected component Union-find Union-find algorithms – see section 9.2 algorithms – see section 9.2
- 12. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 12 Minimum spanning tree vs. Steiner tree Minimum spanning tree vs. Steiner tree c d b a 1 1 1 1 c d b a vs
- 13. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 13 Shortest paths – Dijkstra’s algorithm Shortest paths – Dijkstra’s algorithm Single Source Shortest Paths Problem Single Source Shortest Paths Problem: Given a weighted : Given a weighted connected graph G, find shortest paths from source vertex connected graph G, find shortest paths from source vertex s s to each of the other vertices to each of the other vertices Dijkstra’s algorithm Dijkstra’s algorithm: Similar to Prim’s MST algorithm, with : Similar to Prim’s MST algorithm, with a different way of computing numerical labels: Among vertices a different way of computing numerical labels: Among vertices not already in the tree, it finds vertex not already in the tree, it finds vertex u u with the smallest with the smallest sum sum d dv v + + w w( (v v, ,u u) ) where where v v is a vertex for which shortest path has been already found is a vertex for which shortest path has been already found on preceding iterations (such vertices form a tree) on preceding iterations (such vertices form a tree) d dv v is the length of the shortest path form source to is the length of the shortest path form source to v v w w( (v v, ,u u) is the length (weight) of edge from ) is the length (weight) of edge from v v to to u u
- 14. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 14 Example Example d 4 Tree vertices Remaining vertices Tree vertices Remaining vertices a(-,0) b(a,3) c(-,∞) d(a,7) e(-,∞) a b 4 e 3 7 6 2 5 c a b d 4 c e 3 7 4 6 2 5 a b d 4 c e 3 7 4 6 2 5 a b d 4 c e 3 7 4 6 2 5 b(a,3) c(b,3+4) d(b,3+2) e(-,∞) d(b,5) c(b,7) e(d,5+4) c(b,7) e(d,9) e(d,9) d a b d 4 c e 3 7 4 6 2 5
- 15. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 15 Notes on Dijkstra’s algorithm Notes on Dijkstra’s algorithm Doesn’t work for graphs with negative weights Doesn’t work for graphs with negative weights Applicable to both undirected and directed graphs Applicable to both undirected and directed graphs Efficiency Efficiency • O(|V| O(|V|2 2 ) for graphs represented by weight matrix and ) for graphs represented by weight matrix and array implementation of priority queue array implementation of priority queue • O(|E|log|V|) for graphs represented by adj. lists and O(|E|log|V|) for graphs represented by adj. lists and min-heap implementation of priority queue min-heap implementation of priority queue Don’t mix up Dijkstra’s algorithm with Prim’s algorithm! Don’t mix up Dijkstra’s algorithm with Prim’s algorithm!
- 16. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 16 Coding Problem Coding Problem Coding Coding: assignment of bit strings to alphabet characters : assignment of bit strings to alphabet characters Codewords Codewords: bit strings assigned for characters of alphabet : bit strings assigned for characters of alphabet Two types of codes: Two types of codes: fixed-length encoding fixed-length encoding (e.g., ASCII) (e.g., ASCII) variable-length encoding variable-length encoding (e,g., Morse code) (e,g., Morse code) Prefix-free codes Prefix-free codes: no codeword is a prefix of another codeword : no codeword is a prefix of another codeword Problem: If frequencies of the character occurrences are Problem: If frequencies of the character occurrences are known, what is the best binary prefix-free code? known, what is the best binary prefix-free code?
- 17. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 17 Huffman codes Huffman codes Any binary tree with edges labeled with 0’s and 1’s yields a Any binary tree with edges labeled with 0’s and 1’s yields a prefix-free code of characters assigned to its leaves prefix-free code of characters assigned to its leaves Optimal binary tree minimizing the expected (weighted Optimal binary tree minimizing the expected (weighted average) length of a codeword can be constructed as follows average) length of a codeword can be constructed as follows Huffman’s algorithm Huffman’s algorithm Initialize Initialize n n one-node trees with alphabet characters and the tree one-node trees with alphabet characters and the tree weights with their frequencies. weights with their frequencies. Repeat the following step Repeat the following step n n-1 times: join two binary trees with -1 times: join two binary trees with smallest weights into one (as left and right subtrees) and smallest weights into one (as left and right subtrees) and make its weight equal the sum of the weights of the two trees. make its weight equal the sum of the weights of the two trees. Mark edges leading to left and right subtrees with 0’s and 1’s, Mark edges leading to left and right subtrees with 0’s and 1’s, respectively. respectively.
- 18. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 9 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 18 Example Example character A character A B C D B C D _ _ frequency 0.35 0.1 0.2 0.2 0.15 frequency 0.35 0.1 0.2 0.2 0.15 codeword 11 100 00 01 101 codeword 11 100 00 01 101 average bits per character: 2.25 average bits per character: 2.25 for fixed-length encoding: 3 for fixed-length encoding: 3 compression ratio compression ratio: (3-2.25)/3*100% = 25% : (3-2.25)/3*100% = 25% 0.25 0.1 B 0.15 _ 0.2 C 0.2 D 0.35 A 0.2 C 0.2 D 0.35 A 0.1 B 0.15 _ 0.4 0.2 C 0.2 D 0.6 0.25 0.1 B 0.15 _ 0.6 1.0 0 1 0.4 0.2 C 0.2 D 0.25 0.1 B 0.15 _ 0 1 0 0 1 1 0.25 0.1 B 0.15 _ 0.35 A 0.4 0.2 C 0.2 D 0.35 A 0.35 A