Graph processing - Graphlab

GraphLab: A New Framework For Parallel Machine
Learning
Amir H. Payberah
amir@sics.se
Amirkabir University of Technology
(Tehran Polytechnic)
Amir H. Payberah (Tehran Polytechnic) GraphLab 1393/9/8 1 / 42

Reminder

Data-Parallel Model for Large-Scale Graph Processing
The platforms that have worked well for developing parallel applica-
tions are not necessarily eﬀective for large-scale graph problems.

Graph-Parallel Processing
Restricts the types of computation.
New techniques to partition and distribute graphs.
Exploit graph structure.
Executes graph algorithms orders-of-magnitude faster than more
general data-parallel systems.

Data-Parallel vs. Graph-Parallel Computation

Pregel
Vertex-centric

Pregel
Vertex-centric
Bulk Synchronous Parallel (BSP)

Pregel
Vertex-centric
Runs in sequence of iterations (supersteps)

Pregel
Vertex-centric
Runs in sequence of iterations (supersteps)
A vertex in superstep S can:
• reads messages sent to it in superstep S-1.
• sends messages to other vertices: receiving at superstep S+1.
• modiﬁes its state.

Pregel Limitations
Inefficient if different regions of the graph converge at different
speed.
Can suffer if one task is more expensive than the others.
Runtime of each phase is determined by the slowest machine.

Data Model
A directed graph that stores the program state, called data graph.

Vertex Scope
The scope of vertex v is the data stored in vertex v, in all adjacent
vertices and adjacent edges.

Programming Model (1/3)
Rather than adopting a message passing as in Pregel, GraphLab
allows the user deﬁned function of a vertex to read and modify any
of the data in its scope.

Update function: user-deﬁned function similar to Compute in Pregel.
Can read and modify the data within the scope of a vertex.
Schedules the future execution of other update functions.

Sync function: similar to aggregate in Pregel.
Maintains global aggregates.
Performs periodically in the background.

Execution Model

Execution Model
Each task in the set of tasks T , is a tuple (f, v) consisting of an
update function f and a vertex v.

Execution Model
Each task in the set of tasks T , is a tuple (f, v) consisting of an
update function f and a vertex v.
After executing an update function (f, g, · · ·) the modiﬁed scope
data in Sv is written back to the data graph.

Example: PageRank
GraphLab_PageRank(i)
// compute sum over neighbors
total = 0
foreach(j in in_neighbors(i)):
total = total + R[j] * wji
// update the PageRank
R[i] = 0.15 + total
// trigger neighbors to run again
foreach(j in out_neighbors(i)):
signal vertex-program on j
R[i] = 0.15 +
j∈Nbrs(i)
wjiR[j]

Data Consistency (1/3)
Overlapped scopes: race-condition in simultaneous execution of two
update functions.

Overlapped scopes: race-condition in simultaneous execution of two
update functions.
Full consistency: during the execution f(v), no other function reads
or modiﬁes data within the v scope.

Edge consistency: during the execution f(v), no other function
reads or modiﬁes any of the data on v or any of the edges adja-
cent to v.

Vertex consistency: during the execution f(v), no other function
will be applied to v.

Sequential Consistency (1/2)
Proving the correctness of a parallel algorithm: sequential consistency

Proving the correctness of a parallel algorithm: sequential consistency
Sequential consistency: if for every parallel execution, there exists a
sequential execution of update functions that produces an equivalent
result.

A simple method to achieve serializability is to ensure that the scopes
of concurrently executing update functions do not overlap.

• The full consistency model is used.

• The edge consistency model is used and update functions do not modify
data in adjacent vertices.

• The edge consistency model is used and update functions do not modify
data in adjacent vertices.
• The vertex consistency model is used and update functions only access
local vertex data.

Consistency vs. Parallelism
Consistency vs. Parallelism
[Low, Y., GraphLab: A Distributed Abstraction for Large Scale Machine Learning (Doctoral dissertation, University of
California), 2013.]

GraphLab Implementation
Shared memory implementation
Distributed implementation

Tasks Schedulers (1/2)
In what order should the tasks (vertex-update function pairs) be
called?

In what order should the tasks (vertex-update function pairs) be
called?
• A collection of base schedules, e.g., round-robin, and synchronous.
• Set scheduler: enables users to compose custom update schedules.

How to add new task in the queue?

How to add new task in the queue?
• FIFO: only permits task creation but do not permit task reordering.
• Prioritized: permits task reordering at the cost of increased overhead.

Consistency
Implemented in C++ using PThreads for parallelism.
Consistency: read-write lock

Consistency
Implemented in C++ using PThreads for parallelism.
Consistency: read-write lock
Vertex consistency
• Central vertex (write-lock)
Edge consistency
• Adjacent vertices (read-locks)
Full consistency
• Central vertex (write-locks)
• Adjacent vertices (write-locks)
Deadlocks are avoided by acquiring locks sequentially following a
canonical order.

GraphLab Implementation
Shared memory implementation
Distributed implementation

Distributed Implementation
Graph partitioning
• How to eﬃciently load, partition and distribute the data graph across
machines?
Consistency
• How to achieve consistency in the distributed setting?
Fault tolerance

Graph Partitioning

Graph Partitioning - Phase 1 (1/2)
Two-phase partitioning.
Partitioning the data graph into k parts, called atom.
• k number of machines

meta-graph: the graph of atoms (one vertex for each atom).

Atom weight: the amount of data it stores.

Atom weight: the amount of data it stores.
Edge weight: the number of edges crossing the atoms.

Each atom is stored as a separate ﬁle on a distributed storage system,
e.g., HDFS.

e.g., HDFS.
Each atom ﬁle is a simple binary that stores interior and the ghosts
of the partition information.

e.g., HDFS.
Each atom ﬁle is a simple binary that stores interior and the ghosts
of the partition information.
Ghost: set of vertices and edges adjacent to the partition boundary.

Graph Partitioning - Phase 2
Meta-graph is very small.
A fast balanced partition of the meta-graph over the physical ma-
chines.
Assigning graph atoms to machines.

Consistency

Consistency
To achieve a serializable parallel execution of a set of dependent
tasks.
• Chromatic engine
• Distributed locking engine

Consistency - Chromatic Engine
Construct a vertex coloring: assigns a color to each vertex such that
no adjacent vertices share the same color.

Edge consistency: executing, synchronously, all update tasks asso-
ciated with vertices of the same color before proceeding to the next
color.

color.
Full consistency: no vertex shares the same color as any of its dis-
tance two neighbors.

color.
Full consistency: no vertex shares the same color as any of its dis-
tance two neighbors.
Vertex consistency: assigning all vertices the same color.

Consistency - Distributed Locking Engine
Associating a readers-writer lock with each vertex.

Vertex consistency

Vertex consistency
Edge consistency
• Central vertex (write-lock), Adjacent vertices (read-locks)

Vertex consistency
Edge consistency
Full consistency
• Central vertex (write-locks), Adjacent vertices (write-locks)

Vertex consistency
Edge consistency
Full consistency
• Central vertex (write-locks), Adjacent vertices (write-locks)
Deadlocks are avoided by acquiring locks sequentially following a
canonical order.

Fault Tolerance

Fault Tolerance - Synchronous
The systems periodically signals all computation activity to halt.

Then synchronizes all caches (ghosts) and saves to disk all data
which has been modiﬁed since the last snapshot.

Then synchronizes all caches (ghosts) and saves to disk all data
which has been modiﬁed since the last snapshot.
Simple, but eliminates the systems advantage of asynchronous com-
putation.

Fault Tolerance - Asynchronous
Based on the Chandy-Lamport algorithm.

The snapshot function is implemented as an update function in
vertices.

vertices.
The snapshot update takes priority over all other update functions.

vertices.
The snapshot update takes priority over all other update functions.
Edge Consistency is used on all update functions.

Summary

GraphLab Summary
Asynchronous model
Vertex-centric
Communication: distributed shared memory
Three consistency levels: full, edge-level, and vertex-level
Partitioning: two-phase partitioning
Consistency: chromatic engine (graph coloring), distributed lock
engine (reader-writer lock)

GraphLab Limitations
Poor performance on Natural graphs.

Questions?

Graph processing - Graphlab

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Graph processing - Graphlab (20)

More from Amir Payberah (13)

Recently uploaded (20)

Graph processing - Graphlab