CCS 3102 Lecture 2_ Mathematical foundations.pdf

Design and Analysis of
Algorithms
Mathematical foundations
Department of Computer Science

Mathematical Background
❖Analysis of algorithms requires knowledge of
mathematical tools such as:
 Methods for evaluating and bounding
summations
 Sets, relations, functions, graphs and trees
 Counting: permutations and combinations
 Probability and statistics
Monday, March 6, 2023
Design and Analysis of Computer Algorithm 2

Sets
❖A set is a collection of distinguishable objects; called
members or elements
❖A set cannot contain the same element more than
once; a variation of this is called multiset
❖Given a set say S and element x then we can write:
❖ ( “x is in S”) OR (“x is not in S”)
❖Frequently used sets
 the empty set
 Z set of integers {…, -2,-1,0,1,2,…}
 R set of real numbers
 N set of natural numbers {0,1,2,…}
S
x S
x 
θ

Set relations / operations
❖Subset A is a subset of B
❖Proper subset if but
❖Intersection
❖Union
❖Difference A-B
B
A 
B
A  B
A  B
A 
B
A
B
A

Why study algorithms and performance?
❖Algorithms help us to understand scalability.
❖Performance often draws the line between what is
feasible and what is impossible.
❖Algorithmic mathematics provides a language for
talking about program behavior.
❖Performance is the currency of computing.
❖The lessons of program performance generalize to
other computing resources.
❖Speed is fun!

Methods of Proof
❖Proof by Contradiction
 Assume a theorem is false; show that this assumption
implies a property known to be true is false -- therefore
original hypothesis must be true
❖Proof by Counterexample
 Use a concrete example to show an inequality cannot
hold
❖Mathematical Induction
 Prove a trivial base case, assume true for k, then show
hypothesis is true for k+1
 Used to prove recursive algorithms
Monday, March 6, 2023 6
Design and Analysis of Computer Algorithm

Review: Induction
❖Suppose
 S(k) is true for fixed constant k
▪ Often k = 0
 S(n) S(n+1) for all n >= k
❖Then S(n) is true for all n >= k

Proof By Induction
❖Claim: S(n) is true for all n >= k
❖Basis:
 Show formula is true when n = k
❖Inductive hypothesis:
 Assume formula is true for an arbitrary n
❖Step:
 Show that formula is then true for n+1

Induction Example:
Gaussian Closed Form
❖Prove 1 + 2 + 3 + … + n = n(n+1) / 2
 Basis:
▪ If n = 0, then 0 = 0(0+1) / 2
 Inductive hypothesis:
▪ Assume 1 + 2 + 3 + … + n = n(n+1) / 2
 Step (show true for n+1):
1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1)
= n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2
= (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2

Induction Example:
Geometric Closed Form
❖Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for
all a  1
 Basis: show that a0 = (a0+1 - 1)/(a - 1)
a0 = 1 = (a1 - 1)/(a - 1)
 Inductive hypothesis:
▪ Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1)
 Step (show true for n+1):
a0 + a1 + … + an+1 = a0 + a1 + … + an + an+1
= (an+1 - 1)/(a - 1) + an+1 = (an+1 – 1+ an+1+1– an+1)/(a - 1)
= (an+1+1 - 1)/(a - 1)

Induction
❖We’ve been using weak induction
❖Strong induction also holds
 Basis: show S(0)
 Hypothesis: assume S(k) holds for arbitrary k <=
n
 Step: Show S(n+1) follows
❖Another variation:
 Basis: show S(0), S(1)
 Hypothesis: assume S(n) and S(n+1) are true
 Step: show S(n+2) follows

Loop Invariants
❖ We use loop invariants to understand why an
algorithm is correct by showing 3 things:
1. Initialisation - it is true prior to the first iteration of
the loop
2. Maintenance – if it is true before an iteration of the
loop, it remains true before the next iteration
3. Termination – when the loop terminates, the
invariant gives us a useful property that helps us
show that the algorithm is correct

Loop Invariants in Use
Insertion Sort
1 for j = 2 to n
2 x = A[j ]
3 i = j − 1
4 while i > 0 and A[i ] > x
5 A[i + 1] = A[i ];
6 i = i − 1
7 A[i +1] = x

Correctness of insertion sort
❖ Each time execution reaches (2), the
elements A[1..j − 1] (i.e. s′) are in sorted
order, and they are the same elements as
were originally in A[1..j − 1] (permuted).
❖ The elements A[j ..n] (i.e. s) are unchanged.
(1) perm(A[1..j − 1],A0[1..j − 1])
(2) ordered(A[1..j − 1])
(3) A[j ..n] = A0[j ..n]
❖ This is a loop invariant for the outer loop.

Using the invariant to prove correctness
❖ Initialization: Show that the loop invariant
holds prior to the first iteration.
 Here, j = 2, so A[1..j − 1] is just the singleton
A[1].
(1) A0[1] = A[1], so the permutation is trivial.
(2) {A[1]} is trivially ordered.
(3) A[2..n] = A0[2..n].

❖ Maintenance: Assume that the loop invariant holds
before some iteration. Show that it will hold again
before the next.
(1) Assume perm(A[1..j − 1],A0[1..j − 1]) and show
perm(A′[1..j ′ − 1],A0[1..j ′ − 1]), where j ′ = j + 1
and A′ is the array A after execution of lines 2–7.
(2) Assume ordered(A[1..j − 1]) and show ordered
(A′[1..j ′ − 1]).
(3) A[j ..n] = A0[j ..n] because we have not yet
touched this part of the array.

❖ Termination: Assume that the loop invariant
holds, and that the loop has terminated.
Show that the postcondition is satisfied.
❖ Here, the loop terminates with j = n + 1.
(1) perm(A[1..j − 1],A0[1..j − 1]) implies
perm(A[1..n],A0[1..n]).
(2) ordered(A[1..j − 1]) implies ordered(A[1..n]).
(3) A[j ..n] is empty.

❖ We should also show that the program actually
does terminate. In this this case, the outer loop
executes exactly n − 1 times.
❖ The inner loop starts with i = j − 1, which is a
positive value; at each iteration, i is decremented;
and the loop exits when i = 0 (or perhaps sooner,
depending on x and A). This guarantees
termination.
❖ The other operations in the algorithm are all
assignments or comparisons, which (we assume)
always terminate.

CCS 3102 Lecture 2_ Mathematical foundations.pdf

More Related Content

Similar to CCS 3102 Lecture 2_ Mathematical foundations.pdf (20)

Recently uploaded (20)

CCS 3102 Lecture 2_ Mathematical foundations.pdf