data unit notes from department of computer science
1. Subject Name:
DESIGN AND ANALYSIS OF
ALGORITHMS
Subject Code:
10CS43
Prepared By:
Sindhuja K
Department:
CSE
Date
06/20/25
2. 06/20/25
OBJECTIVE
• This course aims in introducing recurrence relations, and illustrates their
role in asymptotic and probabilistic analysis of algorithms. It covers in detail
greedy strategies, divide and conquer techniques, dynamic programming and
max flow - min cut theory for designing algorithms, and illustrates them
using a number of well-known problems and applications.
• It also covers popular graph and matching algorithms, and basics of
randomized algorithms and computational complexity.
• Basic knowledge of computational complexity, approximation and
randomized algorithms.
• Teaches how to design efficient algorithms which leads to efficient
programs.
3. 06/20/25
LEARNING OUTCOMES
• Upon completion of this course, students will be able to do
the following:
• Get a solid foundation in algorithm design and analysis.
• Analyze the asymptotic performance of algorithms.
• Write correctness proofs for algorithms.
• Demonstrate a familiarity with major algorithms and data
structures.
• Apply important algorithmic design paradigms and methods of
analysis.
4. Engineered for
Tomorrow
06/20/25
• Text Books:
1. Anany Levitin: Introduction to The Design & Analysis of Algorithms,
2nd Edition, Pearson Education, 2007.
(Listed topics only from the Chapters 1, 2, 3, 5, 7, 8, 10, 11).
2. Ellis Horowitz, Sartaj Sahni, Sanguthevar Rajasekaran: Fundamentals
of Computer Algorithms, 2nd Edition, Universities Press, 2007.
(Listed topics only from the Chapters 3, 4, 5, 13)
• Reference Books:
1. Thomas H. Cormen, Charles E. Leiserson, Ronal L. Rivest, Clifford
Stein: Introduction to Algorithms, 3rd Edition, PHI, 2010.
2. R.C.T. Lee, S.S. Tseng, R.C. Chang & Y.T.Tsai: Introduction to the
Design and Analysis of Algorithms A Strategic Approach, Tata McGraw
Hill, 2005.
5. 06/20/25
COURSE TOPICS
UNIT 1 – INTRODUCTION
UNIT 2 - DIVIDE AND CONQUER
UNIT 3 - THE GREEDY METHOD
UNIT 4 - DYNAMIC PROGRAMMING
UNIT 5 - DECREASE-AND-CONQUER APPROACHES, SPACE-TIME
TRADEOFFS
UNIT 6 - LIMITATIONS OF ALGORITHMIC POWER AND COPING
WITH THEM
UNIT 7 - COPING WITH LIMITATIONS OF ALGORITHMIC POWER
UNIT 8 - PRAM ALGORITHMS
7. 06/20/25
Unit 1- TOPICS
• Notion of Algorithm
• Review of Asymptotic Notations
• Mathematical Analysis of Non-Recursive and Recursive Algorithms
• Brute Force Approaches: Introduction
• Selection Sort and Bubble Sort
• Sequential Search and Brute Force String Matching
8. ALGORITHM
• An algorithm is an exact specification of how to solve a computational
problem
• An algorithm must specify every step completely, so a computer can
implement it without any further “understanding”
• An algorithm must work for all possible inputs of the problem.
• Algorithms must be:
• Correct: For each input produce an appropriate output
• Efficient: run as quickly as possible, and use as little memory as possible
– more about this later
• There can be many different algorithms for each computational problem.
9. WHAT IS AN ALGORITHM?
An algorithm is a sequence of unambiguous instructions
for solving a problem, i.e., for obtaining a required
output for any legitimate input in a finite amount of time.
“computer”
problem
algorithm
input output
Engineered for
Tomorrow
10. Euclid’s algorithm
Step 1 If n = 0, return m and stop; otherwise go to Step 2
Step 2 Divide m by n and assign the value of the remainder to r
Step 3 Assign the value of n to m and the value of r to n. Go to
Step 1.
while n ≠ 0 do
r ← m mod n
m← n
n ← r
return m
Engineered for
Tomorrow
NOTION OF ALGORITHM
11. Problem: Find gcd(m,n), the greatest common divisor of two non-negative, not
both zero integers m and n
Examples: gcd(60,24) = 12, gcd(60,0) = 60
Euclid’s algorithm is based on repeated application of equality gcd(m,n) =
gcd(n, m mod n) until the second number becomes 0, which makes the
problem trivial.
Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
Engineered for
Tomorrow
12. Other methods for computing gcd(m,n)
Consecutive integer checking algorithm
Step 1 Assign the value of min{m,n} to t
Step 2 Divide m by t. If the remainder is 0, go to Step 3;
otherwise, go to Step 4
Step 3 Divide n by t. If the remainder is 0, return t and stop;
otherwise, go to Step 4
Step 4 Decrease t by 1 and go to Step 2
Engineered for
Tomorrow
13. Other methods for gcd(m,n) [cont.]
Middle-school procedure
Step 1 Find the prime factorization of m
Step 2 Find the prime factorization of n
Step 3 Find all the common prime factors
Step 4 Compute the product of all the common prime factors
and return it as gcd(m,n)
Engineered for
Tomorrow
14. Sieve of Eratosthenes
Input: Integer n ≥ 2
Output: List of primes less than or equal to n
for p ← 2 to n do A[p] ← p
for p ← 2 to n do
if A[p] != 0 //p hasn’t been previously eliminated from the list
j ← p* p
while j ≤ n do
A[j] ← 0 //mark element as eliminated
j ← j + p
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
2 3 5 7 9 11 13 15 17 19
2 3 5 7 11 13 17 19
2 3 5 7 11 13 17 19
Engineered for
Tomorrow
15. SOME OF THE IMPORTANT POINTS
•
The non-ambiguity requirement for each step of an algorithm cannot be
compromised.
•
The range of inputs for which an algorithm works has to be specified
carefully.
•
The same algorithm can be represented in several different ways.
•
Several algorithms for solving the same problem exist.
•
Algorithms for same problem can be based on very different ideas and can
solve problem dramatically with different speeds
Engineered for
Tomorrow
16. HOW DO WE COMPARE
ALGORITHMS?
We need to define a number of objective measures.
(1) Compare execution times?
Not good: times are specific to a particular computer !!
(2) Count the number of statements executed?
Not good: number of statements vary with the programming language as
well as the style of the individual programmer.
IDEAL SOLUTION
•
Express running time as a function of the input size n (i.e., f(n)).
•
Compare different functions corresponding to running times.
•
Such an analysis is independent of machine time, programming style, etc.
Engineered for
Tomorrow
17. Example
•
Associate a "cost" with each statement.
•
Find the "total cost“ by finding the total number of times each statement is executed.
Algorithm 1 Cost Algorithm 2 Cost
arr[0] = 0; c1 for(i=0; i<N; i++) c2
arr[1] = 0; c1 arr[i] = 0; c1
arr[2] = 0; c1
... ...
arr[N-1] = 0; c1
----------- -------------
c1+c1+...+c1 = c1 x N (N+1) x c2 + N x c1 = (c2 + c1) x N + c2
Engineered for
Tomorrow
18. 18
Another Example
•
Algorithm 3 Cost
sum = 0; c1
for(i=0; i<N; i++) c2
for(j=0; j<N; j++) c2
sum += arr[i][j]; c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N2
Engineered for
Tomorrow
19. Fundamentals of Algorithmic Problem solving
19
Understand the problem
Decide on computational means
Exact vs approximate solution
Data structures
Algorithm design technique
Design an algorithm
Prove correctness
Analyze the algorithm
Code the algorithm
20. ANALYSIS OF ALGORITHMS
•
How good is the algorithm?
•
Correctness
•
Time efficiency
•
Space efficiency
Does there exist a better algorithm?
•
Lower bounds
•
Optimality
Engineered for
Tomorrow
21. ASYMPTOTIC ANALYSIS
•
To compare two algorithms with running times f(n) and g(n), we need a
rough measure that characterizes how fast each function grows.
•
Hint: use rate of growth
•
Compare functions in the limit, that is, asymptotically!
(i.e., for large values of n)
Engineered for
Tomorrow
29. Establishing order of growth using limits
limT(n)/g(n) =
0
0 order of growth of T
T(
(n)
n) < order of growth of g
g(
(n
n)
)
c
c > 0
> 0 order of growth of T
T(
(n)
n) = order of growth of g
g(
(n
n)
)
∞
∞ order of growth of T
T(
(n)
n) > order of growth of g
g(
(n
n)
)
Examples:
Examples:
• 10
10n
n vs.
vs. n
n2
2
• n
n(
(n
n+1)/2 vs.
+1)/2 vs. n
n2
2
n
n→∞
→∞
30. L’Hôpital’s rule and Stirling’s formula
L’Hôpital’s rule: If limn f(n) = limn g(n) = and
the derivatives f´, g´ exist, then
Stirling’s formula: n! (2n)1/2
(n/e)n
f
f(
(n
n)
)
g
g(
(n
n)
)
lim
lim
n
n
=
f
f ´(
´(n
n)
)
g
g ´(
´(n
n)
)
lim
lim
n
n
Example: log
Example: log n
n vs.
vs. n
n
Example: 2
Example: 2n
n
vs.
vs. n
n!
!
31. Orders of growth of some important
functions
• All logarithmic functions loga n belong to the same
class
(log n) no matter what the logarithm’s base a > 1 is
• All polynomials of the same degree k belong to the
same class: aknk
+ ak-1nk-1
+ … + a0 (nk
)
• Exponential functions an
have different orders of
growth for different a’s
• order log n < order n (>0) < order an
< order n! <
order nn
32. Basic asymptotic efficiency classes
1
1 constant
constant
log
log n
n logarithmic
logarithmic
n
n linear
linear
n
n log
log n
n n-
n-log
log-n
-n
n
n2
2
quadratic
quadratic
n
n3
3
cubic
cubic
2
2n
n
exponential
exponential
n
n!
! factorial
factorial
33. MATHEMATICAL ANALYSIS OF NO
RECURSIVE ALGORITHMS
General Plan for Analysis
• Decide on parameter n indicating input size
• Identify algorithm’s basic operation
• Determine worst, average, and best cases for input of size n
• Set up a sum for the number of times the basic operation is
executed
• Simplify the sum using standard formulas and rules
37. MATHEMATICAL ANALYSIS OF
RECURSIVE ALGORITHMS
• Decide on a parameter indicating an input’s size.
• Identify the algorithm’s basic operation.
• Check whether the number of times the basic op. is executed may vary on
different inputs of the same size. (If it may, the worst, average, and best cases
must be investigated separately.)
• Set up a recurrence relation with an appropriate initial condition expressing the
number of times the basic op. is executed.
• Solve the recurrence (or, at the very least, establish its solution’s order of
growth) by backward substitutions or another method.
38. Example 1: Recursive evaluation of n!
Definition: n ! = 1 2 … (n-1) n for n ≥ 1 and 0! = 1
Recursive definition of n!: F(n) = F(n-1) n for n ≥ 1 and
F(0) = 1
Size:
Basic operation:
Recurrence relation:
39. Recursion
•
To see how the recursion works, let’s break down the
factorial function to solve factorial(3)
Engineered for
Tomorrow
40. Breakdown
•
Here, we see that we start at the top level, factorial(3), and simplify the
problem into 3 x factorial(2).
•
Now, we have a slightly less complicated problem in factorial(2), and we
simplify this problem into 2 x factorial(1).
Engineered for
Tomorrow
41. Breakdown
•
We continue this process until we are able to reach a
problem that has a known solution.
•
In this case, that known solution is factorial(0) = 1.
•
The functions then return in reverse order to complete the
solution.
Engineered for
Tomorrow
42. Analysis of Factorial
• Recurrence Relation
M(n) = M(n-1) + 1
M(0) = 0
• Solve by the method of backward substitutions
M(n) = M(n-1) + 1
= [M(n-2) + 1] + 1 = M(n-2) + 2 substituted M(n-2) for M(n-1)
= [M(n-3) + 1] + 2 = M(n-3) + 3 substituted M(n-3) for M(n-2)
.. a pattern evolves
= M(0) + n
= n
Therefore M(n) ε Θ(n)
06/20/25
44. Analysis of Counting # of bits
• Recursive relation including initial conditions
A(n) = A(floor(n/2)) + 1
IC A(1) = 0
• substitute n = 2k
(also k = lg(n))
A(2k) = A(2k-1) + 1 and IC A(20) = 0
A(2k
) = [A(2k-2
) + 1] + 1 = A(2k-2
) + 2
= [A(2k-3
) + 1] + 2 = A(2k-3
) + 3
...
= A(2k-i) + i
...
= A(2k-k
) + k
A(2k
) = k
Substitute back k = lg(n)
A(n) = lg(n) ε Θ(lg n)
06/20/25
45. Given: Three Pegs A, B and C
Peg A initially has n disks, different size, stacked up,
larger disks are below smaller disks
Problem: to move the n disks to Peg C, subject to
1. Can move only one disk at a time
2. Smaller disk should be above larger disk
3. Can use other peg as intermediate
Example 3: Tower of Hanoi
A B C A B C
46. Tower of Hanoi
• How to Solve: Strategy…
– Generalize first: Consider n disks for all n 1
– Our example is only the case when n=4
• Look at small instances…
– How about n=1
• Of course, just “Move disk 1 from A to C”
– How about n=2?
1. “Move disk 1 from A to B”
2. “Move disk 2 from A to C”
3. “Move disk 1 from B to C”
47. Tower of Hanoi (Solution!)
• General Method:
– First, move first (n-1) disks from A to B
– Now, can move largest disk from A to C
– Then, move first (n-1) disks from B to C
• Try this method for n=3
1. “Move disk 1 from A to C”
2. “Move disk 2 from A to B”
3. “Move disk 1 from C to B”
4. “Move disk 3 from A to C”
5. “Move disk 1 from B to A”
6. “Move disk 1 from B to C”
7. “Move disk 1 from A to C”
48. Algorithm for Towel of Hanoi (recursive)
• Recursive Algorithm
– when (n=1), we have simple case
– Else (decompose problem and make recursive-calls)
Hanoi(n, A, B, C);
(* Move n disks from A to C via B *)
begin
if (n=1) then “Move top disk from A to C”
else (* when n>1 *)
Hanoi (n-1, A, C, B);
“Move top disk from A to C”
Hanoi (n-1, B, C, A);
endif
end;
50. Iteration vs. Recursion
•
After looking at both iterative and recursive methods, it appears
that the recursive method is much longer and more difficult.
•
If that’s the case, then why would we ever use recursion?
•
It turns out that recursive techniques, although more complicated
to solve by hand, are very simple and elegant to implement in a
computer.
Engineered for
Tomorrow
51. BRUTE FORCE
• A straightforward approach, usually based directly on the problem’s
statement and definitions of the concepts involved
• Usually can solve small sized instances of a problem
• A yardstick to compare with more efficient ones
Examples:
1. Computing an
(a > 0, n a nonnegative integer)
2. Computing n!
3. Multiplying two matrices
4. Searching for a key of a given value in a list
52. BRUTE-FORCE SORTING ALGORITHM
Selection Sort Scan the array to find its smallest element and swap
it with the first element. Then, starting with the second element,
scan the elements to the right of it to find the smallest among
them and swap it with the second elements. Generally, on pass i
(0 i n-2), find the smallest element in A[i..n-1] and swap it
with A[i]:
A[0] . . . A[i-1] | A[i], . . . , A[min], . . ., A[n-1]
in their final positions
55. BUBBLE SORT
• Compare adjacent elements and exchange them if out of
order
• Essentially, it bubbles up the largest element to the last
position
A0, … …, Aj <-> Aj+1, … …, An-i-1 | An-i ≤ … ≤ An-1
?
55
56. ALGORITHM BubbleSort(A[0..n-1])
for i <- 0 to n-2 do
for j <- 0 to n-2-i do
if A[j+1] < A[j]
swap A[j] and A[j+1]
Example : 89, 45, 68, 90, 29, 34, 17
C(n) є Θ(n2
) Sworst(n) = C(n)
56
57. SEQUENTIAL SEARCH
ALGORITHM SequentialSearch(A[0..n-1], K)
//Output: index of the first element in A, whose //value is equal to K or -1
if no such element is found
i <- 0
while i < n and A[i] ≠ K do
i <- i+1
if i < n
return i
else
return -1
Input size: n
Basic op: <, ≠
Cworst(n) = n
57
58. BRUTE-FORCE STRING MATCHING
• pattern: a string of m characters to search for
• text: a (longer) string of n characters to search in
• problem: find a substring in the text that matches the pattern
Brute-force algorithm
Step 1 Align pattern at beginning of text
Step 2 Moving from left to right, compare each character of
pattern to the corresponding character in text until
• all characters are found to match (successful search); or
• a mismatch is detected
Step 3 While pattern is not found and the text is not yet
exhausted, realign pattern one position to the right and
repeat Step 2
#11:Euclid’s algorithm is good for introducing the notion of an algorithm because it
makes a clear separation from a program that implements the algorithm.
It is also one that is familiar to most students.
Al Khowarizmi (many spellings possible...) – “algorism” (originally) and then
later “algorithm” come from his name.
#38:Note the difference between the two recurrences. Students often confuse these!
F(n) = F(n-1) n
F(0) = 1
for the values of n!
------------
M(n) =M(n-1) + 1
M(0) = 0
for the number of multiplications made by this algorithm
![Other methods for gcd(m,n) [cont.]
Middle-school procedure
Step 1 Find the prime factorization of m
Step 2 Find the prime factorization of n
Step 3 Find all the common prime factors
Step 4 Compute the product of all the common prime factors
and return it as gcd(m,n)
Engineered for
Tomorrow](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-13-320.jpg)
![Sieve of Eratosthenes
Input: Integer n ≥ 2
Output: List of primes less than or equal to n
for p ← 2 to n do A[p] ← p
for p ← 2 to n do
if A[p] != 0 //p hasn’t been previously eliminated from the list
j ← p* p
while j ≤ n do
A[j] ← 0 //mark element as eliminated
j ← j + p
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
2 3 5 7 9 11 13 15 17 19
2 3 5 7 11 13 17 19
2 3 5 7 11 13 17 19
Engineered for
Tomorrow](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-14-320.jpg)
![Example
•
Associate a "cost" with each statement.
•
Find the "total cost“ by finding the total number of times each statement is executed.
Algorithm 1 Cost Algorithm 2 Cost
arr[0] = 0; c1 for(i=0; i<N; i++) c2
arr[1] = 0; c1 arr[i] = 0; c1
arr[2] = 0; c1
... ...
arr[N-1] = 0; c1
----------- -------------
c1+c1+...+c1 = c1 x N (N+1) x c2 + N x c1 = (c2 + c1) x N + c2
Engineered for
Tomorrow](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-17-320.jpg)
![18
Another Example
•
Algorithm 3 Cost
sum = 0; c1
for(i=0; i<N; i++) c2
for(j=0; j<N; j++) c2
sum += arr[i][j]; c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N2
Engineered for
Tomorrow](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-18-320.jpg)
![Analysis of Factorial
• Recurrence Relation
M(n) = M(n-1) + 1
M(0) = 0
• Solve by the method of backward substitutions
M(n) = M(n-1) + 1
= [M(n-2) + 1] + 1 = M(n-2) + 2 substituted M(n-2) for M(n-1)
= [M(n-3) + 1] + 2 = M(n-3) + 3 substituted M(n-3) for M(n-2)
.. a pattern evolves
= M(0) + n
= n
Therefore M(n) ε Θ(n)
06/20/25](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-42-320.jpg)
![Analysis of Counting # of bits
• Recursive relation including initial conditions
A(n) = A(floor(n/2)) + 1
IC A(1) = 0
• substitute n = 2k
(also k = lg(n))
A(2k) = A(2k-1) + 1 and IC A(20) = 0
A(2k
) = [A(2k-2
) + 1] + 1 = A(2k-2
) + 2
= [A(2k-3
) + 1] + 2 = A(2k-3
) + 3
...
= A(2k-i) + i
...
= A(2k-k
) + k
A(2k
) = k
Substitute back k = lg(n)
A(n) = lg(n) ε Θ(lg n)
06/20/25](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-44-320.jpg)
![Analysis of TOH
Recursive relation for moving n discs
M(n) = M(n-1) + 1 + M(n-1) = 2M(n-1) + 1
IC: M(1) = 1
Solve using backward substitution
M(n) = 2M(n-1) + 1
= 2[2M(n-2) + 1] +1 = 22
M(n-2) + 2+1
=22
[2M(n-3) +1] + 2+1 = 23
M(n-3) + 22
+ 2 + 1
..
M(n) = 2i
M(n-i) + ∑j=0
-i
2j
= 2i
M(n-i) + 2i
-1
...
M(n) = 2n-1
M(n-(n-1)) + 2n-1
-1 = 2n-1
M(1) + 2n-1
-1 = 2n-1
+ 2n-1
-1 = 2n
-1
M(n) ε Θ(2n
)
06/20/25](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-49-320.jpg)
![BRUTE-FORCE SORTING ALGORITHM
Selection Sort Scan the array to find its smallest element and swap
it with the first element. Then, starting with the second element,
scan the elements to the right of it to find the smallest among
them and swap it with the second elements. Generally, on pass i
(0 i n-2), find the smallest element in A[i..n-1] and swap it
with A[i]:
A[0] . . . A[i-1] | A[i], . . . , A[min], . . ., A[n-1]
in their final positions](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-52-320.jpg)
![ALGORITHM BubbleSort(A[0..n-1])
for i <- 0 to n-2 do
for j <- 0 to n-2-i do
if A[j+1] < A[j]
swap A[j] and A[j+1]
Example : 89, 45, 68, 90, 29, 34, 17
C(n) є Θ(n2
) Sworst(n) = C(n)
56](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-56-320.jpg)
![SEQUENTIAL SEARCH
ALGORITHM SequentialSearch(A[0..n-1], K)
//Output: index of the first element in A, whose //value is equal to K or -1
if no such element is found
i <- 0
while i < n and A[i] ≠ K do
i <- i+1
if i < n
return i
else
return -1
Input size: n
Basic op: <, ≠
Cworst(n) = n
57](https://image.slidesharecdn.com/daauniti-250620083356-43b23630/85/data-unit-notes-from-department-of-computer-science-57-320.jpg)
