SlideShare a Scribd company logo
Divide and Conquer
(Merge Sort)
Divide and Conquer
(Merge Sort)
Divide and Conquer
 Recursive in structure
 Divide the problem into sub-problems that are
similar to the original but smaller in size
 Conquer the sub-problems by solving them
recursively. If they are small enough, just solve
them in a straightforward manner.
 Combine the solutions to create a solution to
the original problem
An Example: Merge Sort
Sorting Problem: Sort a sequence of n elements into
non-decreasing order.
 Divide: Divide the n-element sequence to be
sorted into two subsequences of n/2 elements each
 Conquer: Sort the two subsequences recursively
using merge sort.
 Combine: Merge the two sorted subsequences to
produce the sorted answer.
Merge Sort – Example
18 26 32 6 43 15 9 1
18 26 32 6 43 15 9 1
18 26 32 6 43 15 9 1
2618 632 1543 19
18 26 32 6 43 15 9 1
18 26 32 6 43 15 9 1
18 26 326 15 43 1 9
6 18 26 32 1 9 15 43
1 6 9 15 18 26 32 43
18 26
18 26
18 26
32
32
6
6
32 6
18 26 32 6
43
43
15
15
43 15
9
9
1
1
9 1
43 15 9 1
18 26 32 6 43 15 9 1
18 26 632
626 3218
1543 19
1 915 43
16 9 1518 26 32 43
Original Sequence Sorted Sequence
Merge-Sort (A, p, r)
INPUT: a sequence of n numbers stored in array A
OUTPUT: an ordered sequence of n numbers
MergeSort (A, p, r) // sort A[p..r] by divide & conquer
1 if p < r
2 then q ← (p+r)/2
3 MergeSort (A, p, q)
4 MergeSort (A, q+1, r)
5 Merge (A, p, q, r) // merges A[p..q] with A[q+1..r]
Initial Call: MergeSort(A, 1, n)
Procedure Merge
Merge(A, p, q, r)
1 n1 ← q – p + 1
2 n2 ← r – q
3 for i ← 1 to n1
4 do L[i] ← A[p + i – 1]
5 for j ← 1 to n2
6 do R[j] ← A[q + j]
7 L[n1+1] ← ∞
8 R[n2+1] ← ∞
9 i ← 1
10 j ← 1
11 for k ←p to r
12 do if L[i] ≤ R[j]
13 then A[k] ← L[i]
14 i ← i + 1
15 else A[k] ← R[j]
16 j ← j + 1
Sentinels, to avoid having to
check if either subarray is
fully copied at each step.
Input: Array containing
sorted subarrays A[p..q]
and A[q+1..r].
Output: Merged sorted
subarray in A[p..r].
j
Merge – Example
6 8 26 32 1 9 42 43… …A
k
6 8 26 32 1 9 42 43
k k k k k k k
i i i i
∞ ∞
i j j j j
6 8 26 32 1 9 42 43
1 6 8 9 26 32 42 43
k
L R
Correctness of Merge
Merge(A, p, q, r)
1 n1 ← q – p + 1
2 n2 ← r – q
3 for i ← 1 to n1
4 do L[i] ← A[p + i – 1]
5 for j ← 1 to n2
6 do R[j] ← A[q + j]
7 L[n1+1] ← ∞
8 R[n2+1] ← ∞
9 i ← 1
10 j ← 1
11 for k ←p to r
12 do if L[i] ≤ R[j]
13 then A[k] ← L[i]
14 i ← i + 1
15 else A[k] ← R[j]
16 j ← j + 1
Loop Invariant for the for loop
At the start of each iteration of the
for loop:
Subarray A[p..k – 1]
contains the k – p smallest elements
of L and R in sorted order.
L[i] and R[j] are the smallest elements of
L and R that have not been copied back into
A.
Initialization:
Before the first iteration:
•A[p..k – 1] is empty.
•i = j = 1.
•L[1] and R[1] are the smallest
elements of L and R not copied to A.
Correctness of Merge
Merge(A, p, q, r)
1 n1 ← q – p + 1
2 n2 ← r – q
3 for i ← 1 to n1
4 do L[i] ← A[p + i – 1]
5 for j ← 1 to n2
6 do R[j] ← A[q + j]
7 L[n1+1] ← ∞
8 R[n2+1] ← ∞
9 i ← 1
10 j ← 1
11 for k ←p to r
12 do if L[i] ≤ R[j]
13 then A[k] ← L[i]
14 i ← i + 1
15 else A[k] ← R[j]
16 j ← j + 1
Maintenance:
Case 1: L[i] ≤ R[j]
•By LI, A contains p – k smallest elements
of L and R in sorted order.
•By LI, L[i] and R[j] are the smallest
elements of L and R not yet copied into A.
•Line 13 results in A containing p – k + 1
smallest elements (again in sorted order).
Incrementing i and k reestablishes the LI
for the next iteration.
Similarly for L[i] > R[j].
Termination:
•On termination, k = r + 1.
•By LI, A contains r – p + 1 smallest
elements of L and R in sorted order.
•L and R together contain r – p + 3 elements.
All but the two sentinels have been copied
back into A.
Analysis of Merge Sort
 Running time T(n) of Merge Sort:
 Divide: computing the middle takes Θ(1)
 Conquer: solving 2 subproblems takes 2T(n/2)
 Combine: merging n elements takes Θ(n)
 Total:
T(n) = Θ(1) if n = 1
T(n) = 2T(n/2) + Θ(n) if n > 1
⇒ T(n) = Θ(n lg n) (CLRS, Chapter 4)
Recurrences – IRecurrences – I
Recurrence Relations
 Equation or an inequality that characterizes a
function by its values on smaller inputs.
 Solution Methods (Chapter 4)
 Substitution Method.
 Recursion-tree Method.
 Master Method.
 Recurrence relations arise when we analyze the
running time of iterative or recursive algorithms.
 Ex: Divide and Conquer.
T(n) = Θ(1) if n ≤ c
T(n) = a T(n/b) + D(n) + C(n) otherwise
Substitution Method
 Guess the form of the solution, then
use mathematical induction to show it correct.
 Substitute guessed answer for the function when the
inductive hypothesis is applied to smaller values –
hence, the name.
 Works well when the solution is easy to guess.
 No general way to guess the correct solution.
Example – Exact Function
Recurrence: T(n) = 1 if n = 1
T(n) = 2T(n/2) + n if n > 1
Guess: T(n) = n lg n + n.
Induction:
•Basis: n = 1 ⇒ n lgn + n = 1 = T(n).
•Hypothesis: T(k) = k lg k + k for all k < n.
•Inductive Step: T(n) = 2 T(n/2) + n
= 2 ((n/2)lg(n/2) + (n/2)) + n
= n (lg(n/2)) + 2n
= n lg n – n + 2n
= n lg n + n
Recursion-tree Method
 Making a good guess is sometimes difficult with
the substitution method.
 Use recursion trees to devise good guesses.
 Recursion Trees
 Show successive expansions of recurrences using
trees.
 Keep track of the time spent on the subproblems of a
divide and conquer algorithm.
 Help organize the algebraic bookkeeping necessary
to solve a recurrence.
Recursion Tree – Example
 Running time of Merge Sort:
T(n) = Θ(1) if n = 1
T(n) = 2T(n/2) + Θ(n) if n > 1
 Rewrite the recurrence as
T(n) = c if n = 1
T(n) = 2T(n/2) + cn if n > 1
c > 0: Running time for the base case and
time per array element for the divide and
combine steps.
Recursion Tree for Merge Sort
For the original problem,
we have a cost of cn,
plus two subproblems
each of size (n/2) and
running time T(n/2).
cn
T(n/2) T(n/2)
Each of the size n/2 problems
has a cost of cn/2 plus two
subproblems, each costing
T(n/4).
cn
cn/2 cn/2
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of divide and
merge.
Cost of sorting
subproblems.
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn
cn/2 cn/2
cn/4 cn/4 cn/4 cn/4
c c c cc c
lg n
cn
cn
cn
cn
Total : cnlgn+cn
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn
cn/2 cn/2
cn/4 cn/4 cn/4 cn/4
c c c cc c
•Each level has total cost cn.
•Each time we go down one level,
the number of subproblems doubles,
but the cost per subproblem halves
⇒ cost per level remains the same.
•There are lg n + 1 levels, height is
lg n. (Assuming n is a power of 2.)
•Can be proved by induction.
•Total cost = sum of costs at each
level = (lg n + 1)cn = cnlgn + cn =
Θ(n lgn).
Other Examples
 Use the recursion-tree method to determine a
guess for the recurrences
 T(n) = 3T(n/4) + Θ(n2
).
 T(n) = T(n/3) + T(2n/3) + O(n).
Recursion Trees – Caution Note
 Recursion trees only generate guesses.
 Verify guesses using substitution method.
 A small amount of “sloppiness” can be
tolerated. Why?
 If careful when drawing out a recursion tree and
summing the costs, can be used as direct proof.
The Master Method
 Based on the Master theorem.
 “Cookbook” approach for solving recurrences
of the form
T(n) = aT(n/b) + f(n)
• a ≥ 1, b > 1 are constants.
• f(n) is asymptotically positive.
• n/b may not be an integer, but we ignore floors and
ceilings. Why?
 Requires memorization of three cases.
The Master Theorem
Theorem 4.1
Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and
Let T(n) be defined on nonnegative integers by the recurrence
T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b.
T(n) can be bounded asymptotically in three cases:
1. If f(n) = O(nlogba–ε
) for some constant ε > 0, then T(n) = Θ(nlogba
).
2. If f(n) = Θ(nlogba
), then T(n) = Θ(nlogba
lg n).
3. If f(n) = Ω(nlogba+ε
) for some constant ε > 0,
and if, for some constant c < 1 and all sufficiently large n,
we have a·f(n/b) ≤ c f(n), then T(n) = Θ(f(n)).
Theorem 4.1
Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and
Let T(n) be defined on nonnegative integers by the recurrence
T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b.
T(n) can be bounded asymptotically in three cases:
1. If f(n) = O(nlogba–ε
) for some constant ε > 0, then T(n) = Θ(nlogba
).
2. If f(n) = Θ(nlogba
), then T(n) = Θ(nlogba
lg n).
3. If f(n) = Ω(nlogba+ε
) for some constant ε > 0,
and if, for some constant c < 1 and all sufficiently large n,
we have a·f(n/b) ≤ c f(n), then T(n) = Θ(f(n)).
We’ll return to recurrences as we need them…

More Related Content

PPTX
0 1 knapsack using branch and bound
PPTX
Quick tutorial on IEEE 754 FLOATING POINT representation
PPTX
Matrix chain multiplication
PPTX
The n Queen Problem
PPT
Sum of subsets problem by backtracking 
PPT
Multiplexers & Demultiplexers
PPTX
Turing machine
PPTX
Knapsack problem using greedy approach
0 1 knapsack using branch and bound
Quick tutorial on IEEE 754 FLOATING POINT representation
Matrix chain multiplication
The n Queen Problem
Sum of subsets problem by backtracking 
Multiplexers & Demultiplexers
Turing machine
Knapsack problem using greedy approach

What's hot (20)

DOCX
Best,worst,average case .17581556 045
PPT
Addition and subtraction with signed magnitude data (mano
PPT
Complexity of Algorithm
PPTX
Binary search
PDF
Coin Change Problem
PPTX
Knapsack Problem
PPTX
Basic Traversal and Search Techniques
PPT
Binary Search
PPTX
Priority Queue in Data Structure
PPT
Asymptotic notations
PPTX
Galois field
PPTX
Asymptotic notations(Big O, Omega, Theta )
PDF
Echelon forms
PPTX
Job sequencing with deadlines(with example)
PPTX
Unsigned multiplication
PPT
Unit 1(stld)
PPTX
Regular Expression to Finite Automata
PPT
Dinive conquer algorithm
PPTX
Space complexity
PDF
Bit pair recoding
Best,worst,average case .17581556 045
Addition and subtraction with signed magnitude data (mano
Complexity of Algorithm
Binary search
Coin Change Problem
Knapsack Problem
Basic Traversal and Search Techniques
Binary Search
Priority Queue in Data Structure
Asymptotic notations
Galois field
Asymptotic notations(Big O, Omega, Theta )
Echelon forms
Job sequencing with deadlines(with example)
Unsigned multiplication
Unit 1(stld)
Regular Expression to Finite Automata
Dinive conquer algorithm
Space complexity
Bit pair recoding
Ad

Viewers also liked (16)

PPTX
Lec 8 constructing tennis tournament
PPTX
Divide and Conquer - Part 1
PPT
Sorting Algorithms
PDF
Analysis and design of algorithms part 4
PDF
Master theorem
PPTX
Merge sort
PPT
03 algorithm properties
PPTX
Merge sort analysis and its real time applications
PPT
DESIGN AND ANALYSIS OF ALGORITHMS
PDF
Quick Sort , Merge Sort , Heap Sort
PPTX
Data Structures - Lecture 8 [Sorting Algorithms]
PPTX
Design and Analysis of Algorithms
PDF
Sorting Algorithms
PPT
Chapter 11 - Sorting and Searching
PPTX
Merj sort
PDF
Resume writing for students and freshers
Lec 8 constructing tennis tournament
Divide and Conquer - Part 1
Sorting Algorithms
Analysis and design of algorithms part 4
Master theorem
Merge sort
03 algorithm properties
Merge sort analysis and its real time applications
DESIGN AND ANALYSIS OF ALGORITHMS
Quick Sort , Merge Sort , Heap Sort
Data Structures - Lecture 8 [Sorting Algorithms]
Design and Analysis of Algorithms
Sorting Algorithms
Chapter 11 - Sorting and Searching
Merj sort
Resume writing for students and freshers
Ad

Similar to Divide and conquer (20)

PPT
5.2 divide and conquer
PPT
03 dc
PPT
Admission in india 2015
PPTX
Divide and Conquer_Binary_Search_Merge_Sort.pptx
PPT
Mergesort
PDF
PPTX
Divided and conqurddddddddddddddfffffe.pptx
PPTX
Divide and Conquer in DAA concept. For B Tech CSE
PPT
1-Chapter One - Analysis of Algorithms.ppt
PPTX
solving_Recurrence_relations_using_methods1.pptx
PPT
Algorithm.ppt
PPTX
Algorithim lec1.pptx
PPTX
Merge sort and quick sort
PDF
Recurrences
PPTX
T2311 - Ch 4_Part1.pptx
PPTX
Merge sort algorithm
PPTX
Algorithms - Rocksolid Tour 2013
PPT
MergesortQuickSort.ppt
PPT
presentation_mergesortquicksort_1458716068_193111.ppt
PPTX
Algorithm Design and Complexity - Course 3
5.2 divide and conquer
03 dc
Admission in india 2015
Divide and Conquer_Binary_Search_Merge_Sort.pptx
Mergesort
Divided and conqurddddddddddddddfffffe.pptx
Divide and Conquer in DAA concept. For B Tech CSE
1-Chapter One - Analysis of Algorithms.ppt
solving_Recurrence_relations_using_methods1.pptx
Algorithm.ppt
Algorithim lec1.pptx
Merge sort and quick sort
Recurrences
T2311 - Ch 4_Part1.pptx
Merge sort algorithm
Algorithms - Rocksolid Tour 2013
MergesortQuickSort.ppt
presentation_mergesortquicksort_1458716068_193111.ppt
Algorithm Design and Complexity - Course 3

More from Vikas Sharma (9)

PPT
Cpp tutorial
PPT
Coloring graphs
PPT
Backtracking
PPT
Backtracking
PPT
Knapsack problem
PPT
Rules and steps for developing a software product (rsfsp) by vikas sharma
PPT
Office automation system for scholl (oasfs) by vikas sharma
PPT
Library and member management system (lamms) by vikas sharma
PPT
Website optimization by vikas sharma
Cpp tutorial
Coloring graphs
Backtracking
Backtracking
Knapsack problem
Rules and steps for developing a software product (rsfsp) by vikas sharma
Office automation system for scholl (oasfs) by vikas sharma
Library and member management system (lamms) by vikas sharma
Website optimization by vikas sharma

Recently uploaded (20)

PDF
Pre independence Education in Inndia.pdf
PDF
Insiders guide to clinical Medicine.pdf
PDF
01-Introduction-to-Information-Management.pdf
PDF
Origin of periodic table-Mendeleev’s Periodic-Modern Periodic table
PDF
VCE English Exam - Section C Student Revision Booklet
PDF
Supply Chain Operations Speaking Notes -ICLT Program
PDF
O5-L3 Freight Transport Ops (International) V1.pdf
PDF
Mark Klimek Lecture Notes_240423 revision books _173037.pdf
PDF
Basic Mud Logging Guide for educational purpose
PPTX
Cell Structure & Organelles in detailed.
PDF
102 student loan defaulters named and shamed – Is someone you know on the list?
PDF
Abdominal Access Techniques with Prof. Dr. R K Mishra
PDF
TR - Agricultural Crops Production NC III.pdf
PDF
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
PPTX
Pharma ospi slides which help in ospi learning
PDF
RMMM.pdf make it easy to upload and study
PPTX
Cell Types and Its function , kingdom of life
PPTX
The Healthy Child – Unit II | Child Health Nursing I | B.Sc Nursing 5th Semester
PPTX
Microbial diseases, their pathogenesis and prophylaxis
Pre independence Education in Inndia.pdf
Insiders guide to clinical Medicine.pdf
01-Introduction-to-Information-Management.pdf
Origin of periodic table-Mendeleev’s Periodic-Modern Periodic table
VCE English Exam - Section C Student Revision Booklet
Supply Chain Operations Speaking Notes -ICLT Program
O5-L3 Freight Transport Ops (International) V1.pdf
Mark Klimek Lecture Notes_240423 revision books _173037.pdf
Basic Mud Logging Guide for educational purpose
Cell Structure & Organelles in detailed.
102 student loan defaulters named and shamed – Is someone you know on the list?
Abdominal Access Techniques with Prof. Dr. R K Mishra
TR - Agricultural Crops Production NC III.pdf
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
Pharma ospi slides which help in ospi learning
RMMM.pdf make it easy to upload and study
Cell Types and Its function , kingdom of life
The Healthy Child – Unit II | Child Health Nursing I | B.Sc Nursing 5th Semester
Microbial diseases, their pathogenesis and prophylaxis

Divide and conquer

  • 1. Divide and Conquer (Merge Sort) Divide and Conquer (Merge Sort)
  • 2. Divide and Conquer  Recursive in structure  Divide the problem into sub-problems that are similar to the original but smaller in size  Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner.  Combine the solutions to create a solution to the original problem
  • 3. An Example: Merge Sort Sorting Problem: Sort a sequence of n elements into non-decreasing order.  Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each  Conquer: Sort the two subsequences recursively using merge sort.  Combine: Merge the two sorted subsequences to produce the sorted answer.
  • 4. Merge Sort – Example 18 26 32 6 43 15 9 1 18 26 32 6 43 15 9 1 18 26 32 6 43 15 9 1 2618 632 1543 19 18 26 32 6 43 15 9 1 18 26 32 6 43 15 9 1 18 26 326 15 43 1 9 6 18 26 32 1 9 15 43 1 6 9 15 18 26 32 43 18 26 18 26 18 26 32 32 6 6 32 6 18 26 32 6 43 43 15 15 43 15 9 9 1 1 9 1 43 15 9 1 18 26 32 6 43 15 9 1 18 26 632 626 3218 1543 19 1 915 43 16 9 1518 26 32 43 Original Sequence Sorted Sequence
  • 5. Merge-Sort (A, p, r) INPUT: a sequence of n numbers stored in array A OUTPUT: an ordered sequence of n numbers MergeSort (A, p, r) // sort A[p..r] by divide & conquer 1 if p < r 2 then q ← (p+r)/2 3 MergeSort (A, p, q) 4 MergeSort (A, q+1, r) 5 Merge (A, p, q, r) // merges A[p..q] with A[q+1..r] Initial Call: MergeSort(A, 1, n)
  • 6. Procedure Merge Merge(A, p, q, r) 1 n1 ← q – p + 1 2 n2 ← r – q 3 for i ← 1 to n1 4 do L[i] ← A[p + i – 1] 5 for j ← 1 to n2 6 do R[j] ← A[q + j] 7 L[n1+1] ← ∞ 8 R[n2+1] ← ∞ 9 i ← 1 10 j ← 1 11 for k ←p to r 12 do if L[i] ≤ R[j] 13 then A[k] ← L[i] 14 i ← i + 1 15 else A[k] ← R[j] 16 j ← j + 1 Sentinels, to avoid having to check if either subarray is fully copied at each step. Input: Array containing sorted subarrays A[p..q] and A[q+1..r]. Output: Merged sorted subarray in A[p..r].
  • 7. j Merge – Example 6 8 26 32 1 9 42 43… …A k 6 8 26 32 1 9 42 43 k k k k k k k i i i i ∞ ∞ i j j j j 6 8 26 32 1 9 42 43 1 6 8 9 26 32 42 43 k L R
  • 8. Correctness of Merge Merge(A, p, q, r) 1 n1 ← q – p + 1 2 n2 ← r – q 3 for i ← 1 to n1 4 do L[i] ← A[p + i – 1] 5 for j ← 1 to n2 6 do R[j] ← A[q + j] 7 L[n1+1] ← ∞ 8 R[n2+1] ← ∞ 9 i ← 1 10 j ← 1 11 for k ←p to r 12 do if L[i] ≤ R[j] 13 then A[k] ← L[i] 14 i ← i + 1 15 else A[k] ← R[j] 16 j ← j + 1 Loop Invariant for the for loop At the start of each iteration of the for loop: Subarray A[p..k – 1] contains the k – p smallest elements of L and R in sorted order. L[i] and R[j] are the smallest elements of L and R that have not been copied back into A. Initialization: Before the first iteration: •A[p..k – 1] is empty. •i = j = 1. •L[1] and R[1] are the smallest elements of L and R not copied to A.
  • 9. Correctness of Merge Merge(A, p, q, r) 1 n1 ← q – p + 1 2 n2 ← r – q 3 for i ← 1 to n1 4 do L[i] ← A[p + i – 1] 5 for j ← 1 to n2 6 do R[j] ← A[q + j] 7 L[n1+1] ← ∞ 8 R[n2+1] ← ∞ 9 i ← 1 10 j ← 1 11 for k ←p to r 12 do if L[i] ≤ R[j] 13 then A[k] ← L[i] 14 i ← i + 1 15 else A[k] ← R[j] 16 j ← j + 1 Maintenance: Case 1: L[i] ≤ R[j] •By LI, A contains p – k smallest elements of L and R in sorted order. •By LI, L[i] and R[j] are the smallest elements of L and R not yet copied into A. •Line 13 results in A containing p – k + 1 smallest elements (again in sorted order). Incrementing i and k reestablishes the LI for the next iteration. Similarly for L[i] > R[j]. Termination: •On termination, k = r + 1. •By LI, A contains r – p + 1 smallest elements of L and R in sorted order. •L and R together contain r – p + 3 elements. All but the two sentinels have been copied back into A.
  • 10. Analysis of Merge Sort  Running time T(n) of Merge Sort:  Divide: computing the middle takes Θ(1)  Conquer: solving 2 subproblems takes 2T(n/2)  Combine: merging n elements takes Θ(n)  Total: T(n) = Θ(1) if n = 1 T(n) = 2T(n/2) + Θ(n) if n > 1 ⇒ T(n) = Θ(n lg n) (CLRS, Chapter 4)
  • 12. Recurrence Relations  Equation or an inequality that characterizes a function by its values on smaller inputs.  Solution Methods (Chapter 4)  Substitution Method.  Recursion-tree Method.  Master Method.  Recurrence relations arise when we analyze the running time of iterative or recursive algorithms.  Ex: Divide and Conquer. T(n) = Θ(1) if n ≤ c T(n) = a T(n/b) + D(n) + C(n) otherwise
  • 13. Substitution Method  Guess the form of the solution, then use mathematical induction to show it correct.  Substitute guessed answer for the function when the inductive hypothesis is applied to smaller values – hence, the name.  Works well when the solution is easy to guess.  No general way to guess the correct solution.
  • 14. Example – Exact Function Recurrence: T(n) = 1 if n = 1 T(n) = 2T(n/2) + n if n > 1 Guess: T(n) = n lg n + n. Induction: •Basis: n = 1 ⇒ n lgn + n = 1 = T(n). •Hypothesis: T(k) = k lg k + k for all k < n. •Inductive Step: T(n) = 2 T(n/2) + n = 2 ((n/2)lg(n/2) + (n/2)) + n = n (lg(n/2)) + 2n = n lg n – n + 2n = n lg n + n
  • 15. Recursion-tree Method  Making a good guess is sometimes difficult with the substitution method.  Use recursion trees to devise good guesses.  Recursion Trees  Show successive expansions of recurrences using trees.  Keep track of the time spent on the subproblems of a divide and conquer algorithm.  Help organize the algebraic bookkeeping necessary to solve a recurrence.
  • 16. Recursion Tree – Example  Running time of Merge Sort: T(n) = Θ(1) if n = 1 T(n) = 2T(n/2) + Θ(n) if n > 1  Rewrite the recurrence as T(n) = c if n = 1 T(n) = 2T(n/2) + cn if n > 1 c > 0: Running time for the base case and time per array element for the divide and combine steps.
  • 17. Recursion Tree for Merge Sort For the original problem, we have a cost of cn, plus two subproblems each of size (n/2) and running time T(n/2). cn T(n/2) T(n/2) Each of the size n/2 problems has a cost of cn/2 plus two subproblems, each costing T(n/4). cn cn/2 cn/2 T(n/4) T(n/4) T(n/4) T(n/4) Cost of divide and merge. Cost of sorting subproblems.
  • 18. Recursion Tree for Merge Sort Continue expanding until the problem size reduces to 1. cn cn/2 cn/2 cn/4 cn/4 cn/4 cn/4 c c c cc c lg n cn cn cn cn Total : cnlgn+cn
  • 19. Recursion Tree for Merge Sort Continue expanding until the problem size reduces to 1. cn cn/2 cn/2 cn/4 cn/4 cn/4 cn/4 c c c cc c •Each level has total cost cn. •Each time we go down one level, the number of subproblems doubles, but the cost per subproblem halves ⇒ cost per level remains the same. •There are lg n + 1 levels, height is lg n. (Assuming n is a power of 2.) •Can be proved by induction. •Total cost = sum of costs at each level = (lg n + 1)cn = cnlgn + cn = Θ(n lgn).
  • 20. Other Examples  Use the recursion-tree method to determine a guess for the recurrences  T(n) = 3T(n/4) + Θ(n2 ).  T(n) = T(n/3) + T(2n/3) + O(n).
  • 21. Recursion Trees – Caution Note  Recursion trees only generate guesses.  Verify guesses using substitution method.  A small amount of “sloppiness” can be tolerated. Why?  If careful when drawing out a recursion tree and summing the costs, can be used as direct proof.
  • 22. The Master Method  Based on the Master theorem.  “Cookbook” approach for solving recurrences of the form T(n) = aT(n/b) + f(n) • a ≥ 1, b > 1 are constants. • f(n) is asymptotically positive. • n/b may not be an integer, but we ignore floors and ceilings. Why?  Requires memorization of three cases.
  • 23. The Master Theorem Theorem 4.1 Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases: 1. If f(n) = O(nlogba–ε ) for some constant ε > 0, then T(n) = Θ(nlogba ). 2. If f(n) = Θ(nlogba ), then T(n) = Θ(nlogba lg n). 3. If f(n) = Ω(nlogba+ε ) for some constant ε > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) ≤ c f(n), then T(n) = Θ(f(n)). Theorem 4.1 Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases: 1. If f(n) = O(nlogba–ε ) for some constant ε > 0, then T(n) = Θ(nlogba ). 2. If f(n) = Θ(nlogba ), then T(n) = Θ(nlogba lg n). 3. If f(n) = Ω(nlogba+ε ) for some constant ε > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) ≤ c f(n), then T(n) = Θ(f(n)). We’ll return to recurrences as we need them…

Editor's Notes

  • #15: Talk about how mathematical induction works.