COSC 3101A - Design and Analysis of Algorithms 10

COSC 3101A - Design and
Analysis of Algorithms
10
BFS, DFS
Topological Sort
Strongly Connected Components
Many of these slides are taken from Monica Nicolescu, Univ. of Nevada, Reno, monica@cs.unr.edu

7/6/2004 Lecture 10 COSC3101A 2
Searching in a Graph
• Graph searching = systematically follow the
edges of the graph so as to visit the vertices of
the graph
• Two basic graph searching algorithms:
– Breadth-first search
– Depth-first search
– The difference between them is in the order in which
they explore the unvisited edges of the graph
• Graph algorithms are typically elaborations of
the basic graph-searching algorithms

7/6/2004 Lecture 10 COSC3101A 3
Breadth-First Search (BFS)
• Input:
– A graph G = (V, E) (directed or undirected)
– A source vertex s  V
• Goal:
– Explore the edges of G to “discover” every vertex
reachable from s, taking the ones closest to s first
• Output:
– d[v] = distance (smallest # of edges) from s to v, for
all v  V
– A “breadth-first tree” rooted at s that contains all
reachable vertices

7/6/2004 Lecture 10 COSC3101A 4
Breadth-First Search (cont.)
• Discover vertices in increasing order of distance
from the source s – search in breadth not depth
– Find all vertices at 1 edge from s, then all vertices at 2
edges from s, and so on
1 2
5 4
3
7
7
9
11
12
6

7/6/2004 Lecture 10 COSC3101A 5
Breadth-First Search (cont.)
• Keeping track of progress:
– Color each vertex in either white,
gray or black
– Initially, all vertices are white
– When being discovered a vertex
becomes gray
– After discovering all its adjacent
vertices the node becomes black
– Use FIFO queue Q to maintain the
set of gray vertices
1 2
5 4
3
1 2
5 4
3
source
1 2
5 4
3

7/6/2004 Lecture 10 COSC3101A 6
Breadth-First Tree
• BFS constructs a breadth-first tree
– Initially contains the root (source vertex s)
– When vertex v is discovered while scanning
the adjacency list of a vertex u  vertex v
and edge (u, v) are added to the tree
– u is the predecessor (parent) of v in the
breadth-first tree
– A vertex is discovered only once  it has at
most one parent
1 2
5 4
3
source

7/6/2004 Lecture 10 COSC3101A 7
BFS Additional Data Structures
• G = (V, E) represented using adjacency lists
• color[u] – the color of the vertex for all u  V
• [u] – predecessor of u
– If u = s (root) or node u has not yet been
discovered  [u] = NIL
• d[u] – the distance from the source s to
vertex u
• Use a FIFO queue Q to maintain the set of
gray vertices
1 2
5 4
3
d=1
=1
d=1
=1
d=2
=5
d=2
=2
source

7/6/2004 Lecture 10 COSC3101A 8
BFS(G, s)
1. for each u  V[G] - {s}
2. do color[u]  WHITE
3. d[u] ← 
4. [u] = NIL
5. color[s]  GRAY
6. d[s] ← 0
7. [s] = NIL
8. Q  
9. Q ← ENQUEUE(Q, s)
Q: s
 0  
   
r s t u
v w x y
  
   
r s t u
v w x y
r s t u
v w x y

7/6/2004 Lecture 10 COSC3101A 9
BFS(G, s)
10. while Q  
11. do u ← DEQUEUE(Q)
12. for each v  Adj[u]
13. do if color[v] = WHITE
14. then color[v] ← GRAY
15. d[v] ← d[u] + 1
16. [v] = u
17. ENQUEUE(Q, v)
18. color[u]  BLACK
 0  
 1  
r s t u
v w x y
Q: w
Q: s
 0  
   
r s t u
v w x y
1 0  
 1  
r s t u
v w x y
Q: w, r

7/6/2004 Lecture 10 COSC3101A 10
Example
1 0  
 1  
r s t u
v w x y
Q: s
 0  
   
r s t u
v w x y
Q: w, r
v w x y
1 0 2 
 1 2 
r s t u
Q: r, t, x
1 0 2 
2 1 2 
r s t u
v w x y
Q: t, x, v
1 0 2 3
2 1 2 
r s t u
v w x y
Q: x, v, u
1 0 2 3
2 1 2 3
r s t u
v w x y
Q: v, u, y
1 0 2 3
2 1 2 3
r s t u
v w x y
Q: u, y
1 0 2 3
2 1 2 3
r s t u
v w x y
Q: y
r s t u
1 0 2 3
2 1 2 3
v w x y
Q: 

7/6/2004 Lecture 10 COSC3101A 11
Analysis of BFS
1. for each u  V - {s}
2. do color[u]  WHITE
3. d[u] ← 
4. [u] = NIL
5. color[s]  GRAY
6. d[s] ← 0
7. [s] = NIL
8. Q  
9. Q ← ENQUEUE(Q, s)
O(V)
(1)

7/6/2004 Lecture 10 COSC3101A 12
Analysis of BFS
10. while Q  
11. do u ← DEQUEUE(Q)
14. then color[v] = GRAY
15. d[v] ← d[u] + 1
16. [v] = u
17. ENQUEUE(Q, v)
18. color[u]  BLACK
(1)
(1)
Scan Adj[u] for all vertices
in the graph
• Each vertex is scanned only
once, when the vertex is
dequeued
• Sum of lengths of all
adjacency lists = (E)
• Scanning operations:
O(E)
• Total running time for BFS = O(V + E)

7/6/2004 Lecture 10 COSC3101A 13
Shortest Paths Property
• BFS finds the shortest-path distance from the
source vertex s  V to each node in the graph
• Shortest-path distance = (s, u)
– Minimum number of edges in any path from s to u
r s t u
1 0 2 3
2 1 2 3
v w x y
source

7/6/2004 Lecture 10 COSC3101A 14
Depth-First Search
• Input:
– G = (V, E) (No source vertex given!)
• Goal:
– Explore the edges of G to “discover” every vertex in V
starting at the most current visited node
– Search may be repeated from multiple sources
• Output:
– 2 timestamps on each vertex:
• d[v] = discovery time
• f[v] = finishing time (done with examining v’s adjacency list)
– Depth-first forest
1 2
5 4
3

7/6/2004 Lecture 10 COSC3101A 15
Depth-First Search
• Search “deeper” in the graph whenever
possible
• Edges are explored out of the most recently
discovered vertex v that still has unexplored
edges
• After all edges of v have been explored, the search
“backtracks” from the parent of v
• The process continues until all vertices reachable from the
original source have been discovered
• If undiscovered vertices remain, choose one of them as a
new source and repeat the search from that vertex
• DFS creates a “depth-first forest”
1 2
5 4
3

7/6/2004 Lecture 10 COSC3101A 16
DFS Additional Data Structures
• Global variable: time-step
– Incremented when nodes are discovered/finished
• color[u] – similar to BFS
– White before discovery, gray while processing and
black when finished processing
• [u] – predecessor of u
• d[u], f[u] – discovery and finish times
GRAY
WHITE BLACK
0 2V
d[u] f[u]
1 ≤ d[u] < f [u] ≤ 2 |V|

7/6/2004 Lecture 10 COSC3101A 17
DFS(G)
1. for each u  V[G]
2. do color[u] ← WHITE
3. [u] ← NIL
4. time ← 0
6. do if color[u] = WHITE
7. then DFS-VISIT(u)
• Every time DFS-VISIT(u) is called, u becomes the
root of a new tree in the depth-first forest
u v w
x y z

7/6/2004 Lecture 10 COSC3101A 18
DFS-VISIT(u)
1. color[u] ← GRAY
2. time ← time+1
3. d[u] ← time
6. then [v] ← u
7. DFS-VISIT(v)
8. color[u] ← BLACK
9. time ← time + 1
10. f[u] ← time
1/
u v w
x y z
u v w
x y z
time = 1
1/ 2/
u v w
x y z

7/6/2004 Lecture 10 COSC3101A 19
Example
1/ 2/
u v w
x y z
1/
u v w
x y z
1/ 2/
3/
u v w
x y z
1/ 2/
4/ 3/
u v w
x y z
1/ 2/
4/ 3/
u v w
x y z
B
1/ 2/
4/5 3/
u v w
x y z
B
1/ 2/
4/5 3/6
u v w
x y z
B
1/ 2/7
4/5 3/6
u v w
x y z
B
1/ 2/7
4/5 3/6
u v w
x y z
B
F

7/6/2004 Lecture 10 COSC3101A 20
Example (cont.)
1/8 2/7
4/5 3/6
u v w
x y z
B
F
1/8 2/7 9/
4/5 3/6
u v w
x y z
B
F
1/8 2/7 9/
4/5 3/6
u v w
x y z
B
F
C
1/8 2/7 9/
4/5 3/6 10/
u v w
x y z
B
F
C
1/8 2/7 9/
4/5 3/6 10/
u v w
x y z
B
F
C
B
1/8 2/7 9/
4/5 3/6 10/11
u v w
x y z
B
F
C
B
1/8 2/7 9/12
4/5 3/6 10/11
u v w
x y z
B
F
C
B
The results of DFS may depend on:
• The order in which nodes are explored
in procedure DFS
• The order in which the neighbors of a
vertex are visited in DFS-VISIT

7/6/2004 Lecture 10 COSC3101A 21
Edge Classification
• Tree edge (reaches a WHITE
vertex):
– (u, v) is a tree edge if v was first
discovered by exploring edge (u, v)
• Back edge (reaches a GRAY
vertex):
– (u, v), connecting a vertex u to an
ancestor v in a depth first tree
– Self loops (in directed graphs) are
also back edges
1/
u v w
x y z
1/ 2/
4/ 3/
u v w
x y z
B

7/6/2004 Lecture 10 COSC3101A 22
Edge Classification
• Forward edge (reaches a BLACK
vertex & d[u] < d[v]):
– Non-tree edges (u, v) that connect a vertex
u to a descendant v in a depth first tree
• Cross edge (reaches a BLACK vertex
& d[u] > d[v]):
– Can go between vertices in same depth-first
tree (as long as there is no ancestor /
descendant relation) or between different
depth-first trees
1/ 2/7
4/5 3/6
u v w
x y z
B
F
1/8 2/7 9/
4/5 3/6
u v w
x y z
B
F
C

7/6/2004 Lecture 10 COSC3101A 23
Analysis of DFS(G)
2. do color[u] ← WHITE
3. [u] ← NIL
4. time ← 0
6. do if color[u] = WHITE
7. then DFS-VISIT(u)
(V)
(V) – exclusive
of time for
DFS-VISIT

7/6/2004 Lecture 10 COSC3101A 24
Analysis of DFS-VISIT(u)
1. color[u] ← GRAY
2. time ← time+1
3. d[u] ← time
6. then [v] ← u
7. DFS-VISIT(v)
8. color[u] ← BLACK
9. time ← time + 1
10. f[u] ← time
Each loop takes
|Adj[v]|
DFS-VISIT is called exactly
once for each vertex
Total: ΣvV |Adj[v]| + (V) =
(E)
(V + E)

7/6/2004 Lecture 10 COSC3101A 25
Properties of DFS
• u = [v]  DFS-VISIT(v) was called
during a search of u’s adjacency list
• Vertex v is a descendant of vertex u
in the depth first forest  v is
discovered during the time in which
u is gray
1/ 2/
3/
u v w
x y z

7/6/2004 Lecture 10 COSC3101A 26
Parenthesis Theorem
In any DFS of a graph G, for
all u, v, exactly one of the
following holds:
1. [d[u], f[u]] and [d[v], f[v]] are
disjoint, and neither of u and v
is a descendant of the other
2. [d[v], f[v]] is entirely within
[d[u], f[u]] and v is a
descendant of u
3. [d[u], f[u]] is entirely within
[d[v], f[v]] and u is a
descendant of v
3/6 2/9 1/10
4/5 7/8 12/13
u
v
w
x
y z s
11/16
14/15
t
1 2 3 4 5 6 7 8 9 10 13
11 12 14 15 16
s
z
t
v u
y w
x
(s (z (y (x x) y) (w w) z) s) u)
(t (v (u u) t)
Well-formed expression: parenthesis are
properly nested

7/6/2004 Lecture 10 COSC3101A 27
Other Properties of DFS
Corollary
Vertex v is a proper descendant of u
 d[u] < d[v] < f[v] < f[u]
Theorem (White-path Theorem)
In a depth-first forest of a graph G, vertex
v is a descendant of u if and only if at time
d[u], there is a path u  v consisting of
only white vertices.
1/ 2/
u
v
1/8 2/7 9/12
4/5 3/6 10/11
u
v
B
F
C
B

7/6/2004 Lecture 10 COSC3101A 28
Topological Sort
Topological sort of a directed acyclic graph G =
(V, E): a linear order of vertices such that if there
exists an edge (u, v), then u appears before v in
the ordering.
• Directed acyclic graphs (DAGs)
– Used to represent precedence of events or processes
that have a partial order
a before b b before c
b before c a before c
a before c
What about
a and b?
Topological sort helps us establish a total order

7/6/2004 Lecture 10 COSC3101A 29
Topological Sort
undershorts
pants
belt
socks
shoes
watch
shirt
tie
jacket
TOPOLOGICAL-SORT(V, E)
1. Call DFS(V, E) to compute
finishing times f[v] for each
vertex v
2. When each vertex is finished,
insert it onto the front of a
linked list
3. Return the linked list of
vertices
1/
2/
3/4
5
6/7
8
9/10
11/
12/
13/14
15
16 17/18
jacket
tie
belt
shirt
watch
shoes
pants
undershorts
socks
Running time: (V + E)

7/6/2004 Lecture 10 COSC3101A 30
Topological Sort
undershorts
pants
belt
socks
shoes
watch
shirt
tie
jacket
1/
2/
3/4
5
6/7
8
9/10
11/
12/
13/14
15
16 17/18
jacket
tie
belt
shirt
watch
shoes
pants
undershorts
socks
Topological sort:
an ordering of vertices along a
horizontal line so that all directed
edges go from left to right.

7/6/2004 Lecture 10 COSC3101A 31
Lemma
A directed graph is acyclic  a DFS on G yields
no back edges.
Proof:
“”: acyclic  no back edge
– Assume back edge  prove cycle
– Assume there is a back edge (u, v)
v is an ancestor of u
 there is a path from v to u in G (v  u)
 v  u + the back edge (u, v) yield a cycle
v
u
(u, v)

7/6/2004 Lecture 10 COSC3101A 32
Lemma
A directed graph is acyclic  a DFS on G yields
no back edges.
Proof:
“”: no back edge  acyclic
– Assume cycle  prove back edge
– Suppose G contains cycle c
– Let v be the first vertex discovered in c, and (u, v) be
the preceding edge in c
– At time d[v], vertices of c form a white path v  u
– u is descendant of v in depth-first forest (by white-path
theorem)
 (u, v) is a back edge
v
u
(u, v)

7/6/2004 Lecture 10 COSC3101A 33
Strongly Connected Components
Given directed graph G = (V, E):
A strongly connected component (SCC) of G
is a maximal set of vertices C  V such that for
every pair of vertices u, v  C, we have both
u  v and v  u.

7/6/2004 Lecture 10 COSC3101A 34
The Transpose of a Graph
• GT = transpose of G
– GT is G with all edges reversed
– GT = (V, ET), ET = {(u, v) : (v, u)  E}
• If using adjacency lists: we can create GT in
(V + E) time
1 2
5 4
3
1 2
5 4
3

7/6/2004 Lecture 10 COSC3101A 35
Finding the SCC
• Observation: G and GT have the same SCC’s
– u and v are reachable from each other in G  they are
reachable from each other in GT
• Idea for computing the SCC of a DAG G = (V, E):
– Make two depth first searches: one on G and one on GT
1 2
5 4
3
1 2
5 4
3

7/6/2004 Lecture 10 COSC3101A 36
STRONGLY-CONNECTED-COMPONENTS(G)
1. call DFS(G) to compute finishing times f[u] for
each vertex u
2. compute GT
3. call DFS(GT), but in the main loop of DFS,
consider vertices in order of decreasing f[u]
(as computed in first DFS)
4. output the vertices in each tree of the depth-
first forest formed in second DFS as a separate
SCC

7/6/2004 Lecture 10 COSC3101A 37
Example
a b c d
e f g h
1/
2/
3/4 6
5/
7
8/
11/
12/
13/ 9
10
14
15
16
f
4
h
6
g
7
d
9
c
10
a
14
e
15
b
16
DFS on the initial graph G
DFS on GT:
• start at b: visit a, e
• start at c: visit d
• start at g: visit f
• start at h
Strongly connected components: C1 = {a, b, e}, C2 = {c, d}, C3 = {f, g}, C4 = {h}
a b c d
e f g h

7/6/2004 Lecture 10 COSC3101A 38
Component Graph
• The component graph GSCC = (VSCC, ESCC):
– VSCC = {v1, v2, …, vk}, where vi corresponds to each
strongly connected component Ci
– There is an edge (vi, vj)  ESCC if G contains a
directed edge (x, y) for some x  Ci and y  Cj
• The component graph is a DAG
a b c d
e f g h
a b e
c d
f g h

7/6/2004 Lecture 10 COSC3101A 39
Lemma 1
Let C and C’ be distinct SCC’s in G
Let u, v  C, and u’, v’  C’
Suppose there is a path u  u’ in G
Then there cannot also be a path v’  v in G.
u
v
u’
v’
Proof
• Suppose there is a path v’  v
• There exists u  u’  v’
• There exists v’  v  u
• u and v’ are reachable from
each other, so they are not in
separate SCC’s: contradiction!
C C’

7/6/2004 Lecture 10 COSC3101A 40
Notations
• Extend notation for d (starting time) and f
(finishing time) to sets of vertices U  V:
– d(U) = minuU { d[u] } (earliest discovery time)
– f(U) = maxuU { f[u] } (latest finishing time)
a b c d
e f g h
1/
2/
3/4 6
5/
7
8/
11/
12/
13/ 9
10
14
15
16
C1 C2
C3 C4
d(C1)
f(C1)
d(C2)
f(C2)
d(C3)
f(C3)
d(C4)
f(C4)
=11
=16
=1
=10
=2
=7
=5
=6

7/6/2004 Lecture 10 COSC3101A 41
Lemma 2
• Let C and C’ be distinct SCCs in a directed
graph G = (V, E). If there is an edge (u, v)  E,
where u  C and v  C’ then f(C) > f(C’).
• Consider C1 and C2, connected by edge (b, c)
a b c d
e f g h
1/
2/
3/4 6
5/
7
8/
11/
12/
13/ 9
10
14
15
16
C1 C2
C3 C4
d(C1)
f(C1)
d(C2)
f(C2)
d(C3)
f(C3)
d(C4)
f(C4)
=11
=16
=1
=10
=3
=7
=5
=6

7/6/2004 Lecture 10 COSC3101A 42
Corollary
• Let C and C’ be distinct SCCs in a directed
graph G = (V, E). If there is an edge (u, v)  ET,
where u  C and v  C’ then f(C) < f(C’).
• Consider C2 and C1, connected by edge (c, b)
C1 = C’ C2 = C
C3 C4
a b c d
e f g h
• Since (c, b)  ET
 (b, c)  E
• From previous
lemma:
f(C1) > f(C2)
f(C’) > f(C)

7/6/2004 Lecture 10 COSC3101A 43
Discussion
f(C) < f(C’)
• Each edge in GT that goes between different
components goes from a component with an
earlier finish time (in the DFS) to one with a later
finish time
C1 = C’ C2 = C
C3 C4
a b c d
e f g h

7/6/2004 Lecture 10 COSC3101A 44
Why does SCC Work?
• When we do the second DFS, on GT, we start with a
component C such that f(C) is maximum (b, in our case)
• We start from b and visit all vertices in C1
• From corollary: f(C) > f(C’) for all C  C’  there are no
edges from C to any other SCCs in GT
 DFS will visit only vertices in C1
 The depth-first tree rooted at b contains exactly the
vertices of C1 C1 C2
C3 C4
a b c d
e f g h
f
4
h
6
g
7
d
9
c
10
a
14
e
15
b
16

7/6/2004 Lecture 10 COSC3101A 45
Why does SCC Work? (cont.)
• The next root chosen in the second DFS is in SCC C2
such that f(C) is maximum over all SCC’s other than C1
• DFS visits all vertices in C2
– the only edges out of C2 go to C1, which we’ve already visited
 The only tree edges will be to vertices in C2
• Each time we choose a new root it can reach only:
– vertices in its own component
– vertices in components already visited
C1 C2
C3 C4
a b c d
e f g h
f
4
h
6
g
7
d
9
c
10
a
14
e
15
b
16

7/6/2004 Lecture 10 COSC3101A 46
Readings
• Chapter 22
• Appendix B

COSC 3101A - Design and Analysis of Algorithms 10

More Related Content

Similar to COSC 3101A - Design and Analysis of Algorithms 10 (20)

Recently uploaded (20)

COSC 3101A - Design and Analysis of Algorithms 10