Lecture-10-CS345A-2023 of Design and Analysis

Design and Analysis of Algorithms
Lecture 10
A generic way to design Greedy Algorithms
1
Algorithms-II : CS345A

FINDING THE LABELED BINARY TREE FOR
2
the optimal prefix codes
RECAP OF LAST LECTURE

Optimal Prefix Coding
Algorithmic Problem: Given a set of alphabets and their frequencies,
compute coding such that
• is prefix coding
• is minimum.
3
, , … ,
Non-decreasing order of frequency
𝒇(𝒂¿¿𝒊)≤𝒇(𝒂¿¿𝒊+𝟏)¿¿

The binary tree of the optimal prefix code
4
0 1
0 1 1
0
.
.
.
Deepest level 𝒂𝟏 𝒂𝟐
1
1
0
𝐀𝐁𝐋 (𝜸 )= ∑
𝑥∈ 𝑨
𝑓 (𝑥).|𝐝𝐞𝐩𝐭𝐡𝐓 (𝑥)|
Theorem: There exists an optimal prefix coding in which and appear as siblings

A generic way to design and analyse a greedy algorithm
P: a given optimization problem
So, all you need is to just design the Greedy step appropriately.
5
𝑨
𝑨 ′
instance of size of problem P
Greedy
step
Opt()
Opt()
𝑨 ′ ′
Greedy
step
…
Base case
Opt()
?
Then can you see an
efficient algorithm to
compute Opt() ?
Suppose the Greedy step is such that
we can efficiently obtain Opt() from Opt().
Yes, a simple recursive algorithm.
Ponder over this inference for
some time.
Isn’t it amazing ?

EXAMPLE
MINIMUM SPANNING TREE
6

instance
7
170
33
35
31
41 81
52
42
50
102
50 30
60
70
54
57
49
49
80
78
63
54
53
44
29
64
42
43

instance
Theorem 1 : There is an MST() with edge (u,v)
8
170
33
35
31
41 81
52
42
50
102
50 30
60
70
54
57
49
49
80
78
63
54
53
44
29
64
42
43
u
v

instance
Theorem 1 : There is an MST() with edge (u,v)
9
170
33
35
31
41 81
52
42
50
102
50 30
60
70
54
57
49
49
80
78
63
54
53
64
42
44
w
43

How to compute instance ?
Let (u,v) be the least weight edge in =(, ). Transform into as follows.
• Remove vertices u and v and add a new vertex w
• For each edge (u,x)ϵ , add edge (w,x) in .
• For each edge (v,x)ϵ , add edge (w,x) in .
• In case of multiple edges between w and x, keep only the lighter weight edge.
10

11
𝑨
𝑨 ′
MST()
MST()
Weight of MST() Weight of MST() w(u,v)
Homework
Greedy
step Using Theorem 1
How ?
In the rest of the lecture, we shall discuss a technique
which will also help you do this homework.

FINDING THE LABELED BINARY TREE FOR
12
the optimal prefix codes

13
0 1
0 1 1
0
.
.
.
1
1
0
Theorem 2: There exists an optimal prefix coding in which and appear as siblings
From this picture, can you
design the greedy step that
you will perform to compute
the smaller instance ?

14
0 1
0 1 1
0
.
.
.
Deepest level ’
1

+
Note: Establishing this relation will lead to
an efficient algorithm to extract Opt() from Opt()
15
𝑨
𝑨 ′
, , , … …
Greedy
step
, … …
𝒂′
?
Opt()
Opt()
Relation ?

Proof for
+
Spend some time thinking about the proof
before moving ahead.

16

How to prove
+ ?
Question: Can we derive a prefix coding for from ?
Question: Can we derive a prefix coding for from ?
17

A prefix coding for from
: the binary tree corresponding to
18
.
.
.
0 1
0 1 1
0
𝑻 ′
’
1

This gives a prefix coding for with = ??
 +
19
0 1
0 1 1
0
.
.
.
1
’
𝑻 ′
𝒂𝟏 𝒂𝟐
1
0
+

20
.
.
.
1
1
0
0 1
0 1 1
0
𝑻
Using
Theorem 2


21
.
.
.
Deepest level
1
𝒂𝟏 𝒂𝟐
1
0
0 1
0 1 1
0
𝑻
Using
Theorem 2
’
𝐎𝐏 𝐓𝐀𝐁𝐋 (𝑨)− 𝒇 (𝒂𝟏)− 𝒇 (𝒂𝟐)

We proved
Using (1) and (2)
22
(1)
(2)
Can you see how this
relation can help us extract
Opt() from Opt() ?

 +
23
0 1
0 1 1
0
.
.
.
1
’
𝑻 ′
𝒂𝟏 𝒂𝟐
1
0
+
𝐎𝐏 𝐓𝐀𝐁𝐋 (𝑨)=𝐎𝐏𝐓𝐀𝐁𝐋 ( 𝑨′
)+ 𝒇 (𝒂𝟏)+ 𝒇 (𝒂𝟐)
We have established
¿
So

The algorithm based on
+
OPT()
{ If ||=2, return ;
else
{ Let and be the two alphabets with least frequencies.
Remove and from ;
Create a new alphabet ;
 ;
Insert into ;
T  OPT();
Replace node in T by
return T ;
}
}
24
’ 𝒂𝟏 𝒂𝟐
1
0
𝒂𝟏 𝒂𝟐
1
0
time complexity
Homework:
Achieve time
complexity

based on a suitable Theorem about Opt()
A generic way to design and analyse a greedy algorithm
P: a given optimization problem
1. Try to establish a relation between OPT() and OPT();
2. Try to prove the relation formally by
 deriving a solution of from OPT()
 deriving a solution of from OPT() 25
𝑨
𝑨 ′
Greedy
step
Opt()
Opt()
use Theorem
If you succeed,
you have an algorithm
from construction

The 2 problems we discussed
Synchronizing the delays in a circuit
Scheduling jobs to minimize maximum
lateness
27
5
6
2
2
3 2
6
2 1 2 1
4
2 3
Electric signal
𝟎 time
Design and analyse a greed algorithm based on the generic technique.

Notations and Terminologies
A directed graph
• Represented as Adjacency lists or Adjacency matrix
• ,
Question: what is a path in ?
Answer: A sequence , ,…, such that (,) for all .
Length of a path =
29
𝒗 𝟏 𝒗 𝟐 𝒗 𝟑 𝒗𝒌 −𝟏 𝒗 𝒌
…
12 3 29
No vertex is repeated

Notations and Terminologies
Definition: The path from to of minimum length is called the shortest path
from to
Definition: Distance from to is the length of the shortest path from to .
Notations:
: distance from to .
: The shortest path from to .
30

Problem Definition
Input: A directed graph with and a source vertex
Aim:
• Compute for all
• Compute for all
This problem is simple and beautiful enough to convince anyone about
the importance of designing efficient algorithms.
31

An example to get
an insight into this problem
Inference:
The distance to any vertex depends upon global parameters.
32
𝒔
G
𝒒
12
6
16
11
9
4
7
5

An example to get
Question: Is there any vertex in this picture for which you are certain about
the distance from ?
Answer: vertex .
33
𝒔
𝒖
𝒙
𝒚
𝒛
𝒗
3
5
12
10
7
G
Give reasons.

An example to get
 The shortest path to vertex is edge (,).
34
𝒔
𝒖
𝒙
𝒚
𝒛
𝒗
3
5
12
10
7
G
¿ 𝟎

Designing a greedy algorithm for shortest paths
Establish a relation between shortest paths in
and
shortest paths in
35
𝑮
𝑮 ′
instance of size
Greedy
step
instance of size
How will look like ?
Do it as Homework

Lecture-10-CS345A-2023 of Design and Analysis

More Related Content

Similar to Lecture-10-CS345A-2023 of Design and Analysis (20)

More from ssuser9183b6 (8)

Recently uploaded (20)

Lecture-10-CS345A-2023 of Design and Analysis

Editor's Notes