Graphs, Min Span Tree and Shortest Path Algo

Graphs: MSTs
and Shortest Paths
David Kauchak
cs161
Summer 2009

Administrative
 Grading
 final grade
 extra credit
 TA issues/concerns
 HW6
 shorter
 will be due Wed. 8/12 before class
 Errors in class slides and notes
 Anonymized scores posted

Shortest paths
 What is the shortest path from a to d?
A
B
C E
D

Shortest paths
 BFS
A
B
C E
D

Shortest paths
 What is the shortest path from a to d?
A
B
C E
D
1
1
3
2
2
3
4

Shortest paths
 We can still use BFS
A
B
C E
D
1
1
3
2
2
3
4

Shortest paths
A
B
C E
D
1
1
3
2
2
3
4
A
B
C E
D

Shortest paths
A
B
C E
D

Shortest paths
 What is the problem?
A
B
C E
D

Shortest paths
 Running time is dependent on the weights
A
B
C
4
1
2
A
B
C
200
50
100

Shortest paths
A
B
C
200
50
100
A
B
C

Shortest paths
A
B
C
 Nothing will change as we expand the frontier
until we’ve gone out 100 levels

Dijkstra’s algorithm
prev keeps track of
the shortest path

Single source shortest paths
 All of the shortest path algorithms we’ll look
at today are call “single source shortest
paths” algorithms
 Why?

A
B
C E
D
1
1
3
3
2
1
4
 




A
B
C E
D
1
1
3
3
2
1
4
 


0
Heap
A 0
B 
C 
D 
E 

A
B
C E
D
1
1
3
3
2
1
4
 


0
Heap
B 
C 
D 
E 

A
B
C E
D
1
1
3
3
2
1
4
 

1
0
Heap
C 1
B 
D 
E 

A
B
C E
D
1
1
3
3
2
1
4
3 

1
0
Heap
C 1
B 3
D 
E 

3
A
B
C E
D
1
1
3
2
1
4
3 

1
0
Heap
B 3
D 
E 

3
A
B
C E
D
1
1
3
2
1
4
2 

1
0
Heap
B 2
D 
E 

3
A
B
C E
D
1
1
3
2
1
4
2 
5
1
0
Heap
B 2
E 5
D 

3
A
B
C E
D
1
1
3
2
1
4
2 
5
1
0
Heap
B 2
E 5
D 
Frontier?

3
A
B
C E
D
1
1
3
2
1
4
2 
5
1
0
Heap
B 2
E 5
D 
All nodes reachable
from starting node
within a given distance

3
A
B
C E
D
1
1
3
2
1
4
2 5
3
1
0
Heap
E 3
D 5

3
A
B
C E
D
1
1
3
2
1
4
2 5
3
1
0
Heap
D 5

3
A
B
C E
D
1
1
3
2
1
4
2 5
3
1
0
Heap

A
B
C E
D
1
1
1
2 5
3
1
0
Heap
3

Is Dijkstra’s algorithm correct?
 Invariant:

 Invariant: For every vertex removed from the heap,
dist[v] is the actual shortest distance from s to v

 Invariant: For every vertex removed from the
heap, dist[v] is the actual shortest distance
from s to v
 The only time a vertex gets visited is when the
distance from s to that vertex is smaller than the
distance to any remaining vertex
 Therefore, there cannot be any other path that
hasn’t been visited already that would result in a
shorter path

Running time?
1 call to MakeHeap

Running time?
 Depends on the heap implementation
1 MakeHeap |V| ExtractMin |E| DecreaseKey Total
Array O(|V|) O(|V|2
) O(|E|) O(|V|2
)
Bin heap O(|V|) O(|V| log |V|) O(|E| log |V|) O((|V|+|E|) log |V|)
O(|E| log |V|)

Running time?
Array O(|V|) O(|V|2
) O(|E|) O(|V|2
)
O(|E| log |V|)
Is this an improvement? If |E| < |V|2
/ log |V|

Running time?
Array O(|V|) O(|V|2
) O(|E|) O(|V|2
)
Fib heap O(|V|) O(|V| log |V|) O(|E|) O(|V| log |V| + |E|)
O(|E| log |V|)

What about Dijkstra’s on…?
A
B
C E
D
1
1
-10
5
10

What about Dijkstra’s on…?
A
B
C E
D
1
1
5
10
Dijkstra’s algorithm only works
for positive edge weights

Bounding the distance
 Another invariant: For each vertex v, dist[v]
is an upper bound on the actual shortest
distance
 start of at 
 only update the value if we find a shorter distance
 An update procedure
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 


 Can we ever go wrong applying this update
rule?
 We can apply this rule as many times as we want
and will never underestimate dist[v]
 When will dist[v] be right?
 If u is along the shortest path to v and dist[u] is
correct
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 


 dist[v] will be right if u is along the shortest
path to v and dist[u] is correct
 Consider the shortest path from s to v
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk

 What happens if we update all of the vertices
with the above update?
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk

)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk
correct

)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk
correct correct

 Does the order that we update the vertices
matter?
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk
correct correct

 dist[v] will be right if u is along the shortest path to v
and dist[u] is correct
 How many times do we have to do this for vertex pi
to have the correct shortest path from s?
 i times
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk

 i times
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk
correct correct

 i times
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk
correct correct correct

 i times
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk
correct correct correct correct

 i times
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk
correct correct correct correct …

 What is the longest (vetex-wise) the path from
s to any node v can be?
 |V| - 1 edges/vertices
)}
,
(
]
[
],
[
min{
]
[ v
u
w
u
dist
v
dist
v
dist 

s p1 v
p2 p3 pk
correct correct correct correct …

Bellman-Ford algorithm
Initialize all the
distances

iterate over all
edges/vertices
and apply update
rule

check for negative
cycles

Negative cycles
A
B
C E
D
1
1
-10
5
10
3
What is the shortest path
from a to e?

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
How many edges is
the shortest path
from s to:
A:

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
How many edges is
the shortest path
from s to:
A: 3

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
How many edges is
the shortest path
from s to:
A: 3
B:

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
How many edges is
the shortest path
from s to:
A: 3
B: 5

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
How many edges is
the shortest path
from s to:
A: 3
B: 5
D:

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
How many edges is
the shortest path
from s to:
A: 3
B: 5
D: 7

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
0 






Iteration: 0

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
0 10





8
Iteration: 1

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
0 10



12
9
8
Iteration: 2

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
0 5
10


8
9
8
Iteration: 3
A has the correct
distance and path

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
0 5
6
11

7
9
8
Iteration: 4

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
0 5
5
7
14
7
9
8
Iteration: 5
B has the correct
distance and path

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
0 5
5
6
10
7
9
8
Iteration: 6

G
S
F
E
A
D
B
C
10
8
1
-1
-1
3
1
1
2
-2
-4
0 5
5
6
9
7
9
8
Iteration: 7
D (and all other
nodes) have the
correct distance
and path

Correctness of Bellman-Ford
 Loop invariant:

Correctness of Bellman-Ford
 Loop invariant: After iteration i, all vertices with
shortest paths from s of length i edges or less have
correct distances

Runtime of Bellman-Ford
O(|V| |E|)

Runtime of Bellman-Ford
Can you modify the algorithm to run
faster (in some circumstances)?

All pairs shortest paths
 Simple approach
 Call Bellman-Ford |V| times
 O(|V|2
|E|)
 Floyd-Warshall – Θ(|V|3
)
 Johnson’s algorithm – O(|V|2
log |V| + |V| |E|)

Minimum spanning trees
 What is the lowest weight set of edges that
connects all vertices of an undirected graph
with positive weights
 Input: An undirected, positive weight graph,
G=(V,E)
 Output: A tree T=(V,E’) where E’  E that
minimizes



'
)
(
E
e
e
w
T
weight

MST example
A
B D
C
4
1
2
3
4
F
E
5
4
6
4
A
B D
C
4
1
2
F
E
5
4

MSTs
 Can an MST have a cycle?
A
B D
C
4
1
2
F
E
5
4
4

MSTs
 Can an MST have a cycle?
A
B D
C
4
1
2
F
E
5
4

Applications?
 Connectivity
 Networks (e.g. communications)
 Circuit desing/wiring
 hub/spoke models (e.g. flights,
transportation)
 Traveling salesman problem?

Cuts
 A cut is a partitioning of the vertices into two sets S
and V-S
 An edges “crosses” the cut if it connects a vertex
uV and vV-S
A
B D
C
4
1
2
3
4
F
E
5
4
6
4

Minimum cut property
 Given a partion S, let edge e be the minimum
cost edge that crosses the partition. Every
minimum spanning tree contains edge e.
S V-S
e’
e
Consider an MST with edge e’ that is not the minimum edge

Minimum cut property
 Given a partion S, let edge e be the minimum
cost edge that crosses the partition. Every
minimum spanning tree contains edge e.
S V-S
e’
e
Using e instead of e’, still connects the graph,
but produces a tree with smaller weights

Algorithm ideas?
 Given a partion S, let edge e be the minimum cost
edge that crosses the partition. Every minimum
spanning tree contains edge e.

Kruskal’s algorithm
A
B D
C
4
1
2
3
4
F
E
5
4
6
4
G
MST
A
B D
C
F
E
Add smallest edge that connects
two sets not already connected

A
B D
C
4
1
2
3
4
F
E
5
4
6
4
G
MST
A
B D
C
1
F
E

A
B D
C
4
1
2
3
4
F
E
5
4
6
4
G
MST
A
B D
C
1
2
F
E

A
B D
C
4
1
2
3
4
F
E
5
4
6
4
G
MST
A
B D
C
4
1
2
F
E

A
B D
C
4
1
2
3
4
F
E
5
4
6
4
G
MST
A
B D
C
4
1
2
F
E
4

A
B D
C
4
1
2
3
4
F
E
5
4
6
4
G
MST
A
B D
C
4
1
2
F
E
5
4

Correctness of Kruskal’s
 Never adds an edge that connects already
connected vertices
 Always adds lowest cost edge to connect two sets.
By min cut property, that edge must be part of the
MST

Running time of Kruskal’s
|V| calls to MakeSet

O(|E| log |E|)

2 |E| calls to FindSet

|V| calls to Union

 Disjoint set data structure
O(|E| log |E|) +
MakeSet FindSet
|E| calls
Union
|V| calls
Total
Linked lists |V| O(|V| |E|) |V| O(|V||E| + |E| log |E|)
O(|V| |E|)
Linked lists +
heuristics
|V| O(|E| log |V|) |V| O(|E| log |V|+ |E| log |E|)
O(|E| log |E| )

Prim’s algorithm
 Start at some root node and build out the MST by
adding the lowest weighted edge at the frontier

Prim’s
A
B D
C
4
1
2
3
4
F
E
5
4
6
4
MST
A
B D
C
F
E

Prim’s
A
B D
C
4
1
2
3
4
F
E
5
4
6
4
MST
A
B D
C
F
E
  
  0

Prim’s
A
B D
C
4
1
2
3
4
F
E
5
4
6
4
MST
A
B D
C
F
E
 4 5
 6 0

Prim’s
A
B D
C
4
1
2
3
4
F
E
5
4
6
4
MST
A
B D
C
F
E
1 4 5
4 2 0

Correctness of Prim’s?
 Can we use the min-cut property?
 Given a partion S, let edge e be the minimum cost
edge that crosses the partition. Every minimum
spanning tree contains edge e.
 Let S be the set of vertices visited so far
 The only time we add a new edge is if it’s the
lowest weight edge from S to V-S

Running time of Prim’s
Θ(|V|)

|V| calls to Extract-Min

|E| calls to Decrease-Key

 Same as Dijksta’s algorithm
Array O(|V|) O(|V|2
) O(|E|) O(|V|2
)
Fib heap O(|V|) O(|V| log |V|) O(|E|) O(|V| log |V| + |E|)
O(|E| log |V|)
Kruskal’s: O(|E| log |E| )

Graphs, Min Span Tree and Shortest Path Algo

More Related Content

Similar to Graphs, Min Span Tree and Shortest Path Algo (20)

Recently uploaded (20)

Graphs, Min Span Tree and Shortest Path Algo