Parallel Graph Analytics

Editor's Notes

• #4: I’ll start with TAO analysis, which provides a useful taxonomy for the graph analytics. We’ll surf now through these categories full of key concepts that we’re going to use later.
• #5: First off, we’ll talk about graph topologies. I’ll go pretty fast through here, since these are part of the course contents. We consider Uniform degree graphs which have node’s degree fairly uniform and I provide as an example this road network. We have then power law graphs, with high CC, low diameter in relation to the dimensions of the graph and the rich get richer property. Finally we have Erdos-Renyi graphs, which basically are random graphs.
• #6: Push – reads the label of the active node and writes on its neighbors Pull – The active node reads the label of its neighbors and updates the active node
• #7: Unordered – These soft priority may not be respected at runtime execution. Topology – every node is processed in each super-step, until a convergence criterion is reached. It may be work-inefficient since there may not be computations that need to be performed at each node. It is then much inefficient in sequential and for large sparse graphs. Despite this, it is very efficient if combined with GPU computing and it’s also quite easy to implement. Data – worklists may lead to a centralization bottleneck problem and at each computation new activities may be created and then added to the worklist.
• #8: And by saying speculative work, I mean it gets work-inefficience. Delta - must be set in an appropriate way (usually as a small value), so that the algorithm is work-efficient (delta = 1) without chocking off parallelism. Buckets – so if delta = 10, we have the first bucket going from 0 to 9, the second from 10-19.
• #9: On the top right corner of the page, we can see the pseudocode of the algorithm with the bold part corresponding to what we are executing and the color represents the actual selected bucket. At the first iteration, we select the initial node and we relax along its light edges.
• #10: So we update the node labels, and we relax again on light edges from the new nodes.
• #11: Here we update the distance of the node from 9 to 8 and at next step we do the same for the node with the label currently set to 10.
• #12: Here we have just one more light node to process for the green bucket.
• #13: We now relax along the heavy edges, which are the bold ones, updating their labels.
• #14: Now we finished the first iteration, so we select the yellow bucket.
• #15: In the next step we have just one light edge to relax along.
• #16: We update then the label after the relaxation and on the next step we’ll relax along the heavy nodes.
• #17: There’s nothing to do, so we simply mark the nodes as processed and we terminate.
• #18: If we see the buckets as worklists we can easily see how new activities are pushed into the worklist. Just a data-driven algorithm can do this. If we were talking about a topology driven algorithm, we would have had all nodes as active nodes and at each iteration we would have applied an operation sweeping over the whole graph, repeating the cycle until the attainment of a convergence criterion.
• #19: An example of data-parallel operation is a map operation would consist of applying an f function to a set nodes, producing a new set with the same cardinality. AMORPHOUS - What matters in this parallel abstraction is the memory model, which defines the semantics of how to read and write in overlapped regions.
• #20: The update of vertices and edges labels can be interpreted as a communication between rounds. We have here an example of a reduction operation applied to the Bellman-Ford algorithm, use to update the node label if there are conflicts during execution. On power-law graphs we have a small diameter and high connectivity, so a large number of processors are kept busy and there are few rounds to synchronize. On road networks we have the opposite situation, so we’ll use a small amount of processors and we’ll have a large number of rounds.
• #21: Autonomous – If a conflict is detected, some of the conflicting activities are automatically aborted, otherwise the activity is committed and the updates get visible. Static – The active nodes can be exectued in parallel without any conflict checking like in a matrix-vector product, that will never produce conflicts. Just in time – preprocess of the input graph to find conflict-free schedules to be executed. This strategy is very strong if combined with topology-driven algorithms. Runtime – Execution in rounds, a set of active nodes is chosen, their neighborhoods are computed and only a set of non-conflicting activities is chosen for execution.
• #22: So let’s start talking about systems. We soon give the names of the most popular systems for graph analytics, where Galois is the the system implemented by the researchers of the article I’ve chosen. In addition to the TAO components we’ve been explaining until now, these systems should have two more properties. Pointer-jumping operations are essential to get better performances during the execution of some algorithms, like Connected-component algorithm. We simply redirect a node with degree 1 to the father of its father like in the figure shown. This can help to iterate faster using Disjoint-set operations. Graph partitioning is essential if we want to compute through a distributed-memory cluster. To reduce the span factor we can partition using edges instead of nodes. This technique is called 2D partitioning.
• #23: The abstractions makes the implementations easier for programmers, but may insert limitations. Some limitations may even inhibit some kinds of algorithm implementations. The abstractions obviously introduce performance penalties if compared to the native implementation.
• #24: In the first study, we can see the resolution of common problems on a power-law graph using different systems and making a comparison with the Native implementation. As we can see Galois has done a good work, being pretty close to the native implementation by showing a performance slower just by 1.2%. We’ll explain later why.
• #25: In this study, we analyze the comparison between Galois and other systems, with varying graph structures. Galois shows better performances on the road network because Galois is the only system with Transactional semantics implemented, and, as we said before, BSP-style semantics are work-inefficient with uniform-degree graphs. We have better performances also on SSSP algorithms, because of the implementation of data-driven algorithms. And we finally have better performances also in the Connected Components test, thanks to the implementation in Galois of pointer-jumping operations.
• #26: Amazon and Netflix - …for their recommender systems. This is possible thanks to the use of collaborative filtering techniques. Even Galois has its own defeats, in fact it cannot operate on distributed-memory systems. (for now)