Fast Billion-scale Graph Computation Using a Bimodal Block Processing Model
0 likes202 views
The document discusses the development of M-Flash, a graph computation framework that employs a bimodal block processing model to achieve high-performance large-graph processing with minimized I/O costs. It examines both the organization of graphs in memory and the efficient processing models for different types of graph structure, aiming to streamline calculations for various algorithms. Results demonstrate the framework's performance across various datasets, indicating its effectiveness in handling billion-scale graphs.
Fast Billion-scale Graph Computation Using a Bimodal Block Processing Model
- 1. Fast Billion-scale Graph Computation Using a Bimodal Block Processing Model Hugo Gualdron1, Robson Cordeiro1, Jose Rodrigues-Jr1, Duen Horng (Polo) Chau 2, Minsuk Kahng2, U Kang3 1University of Sao Paulo, Brazil 2Georgia Institute of Technology, Atlanta, USA 3Seoul National University, Republic of Korea {gualdron,robson,junio}@icmc.usp.br, {polo,kahng}@gatech.edu, ukang@snu.ac.kr Sept/2016 Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 1 / 28
- 2. Summary 1 Introduction 2 Graph Organization 3 Processing Model 4 Programming Model 5 Results 6 Conclusions Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 2 / 28
- 3. Introduction Summary 1 Introduction 2 Graph Organization 3 Processing Model 4 Programming Model 5 Results 6 Conclusions Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 3 / 28
- 4. Introduction Introduction Frameworks for large-graph processing based on secondary memory have shown better or equal performance than distributed frameworks. They are better to solve problems such as PageRank, Connected Components and Triangle Counting. Large-graph processing by minimizing computational resources is very important for the industry. “You can have a second computer once you have shown you know how to use the first one.” Paul Barham Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 4 / 28
- 5. Introduction Contributions M-Flash, the fastest graph computation framework to date. A Bimodal Block Processing Model, an innovation that is able to boost the graph computation by minimizing the I/O cost even further. A flexible and simple programming model to easily implement popular and essential graph algorithm including the first single-machine billion-scale eigensolver. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 5 / 28
- 6. Graph Organization Summary 1 Introduction 2 Graph Organization 3 Processing Model 4 Programming Model 5 Results 6 Conclusions Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 6 / 28
- 7. Graph Organization Graph Organization – Facts Edges and vertex values do not fit in main memory. Sequential operations on disk must be maximized, improving the performance on HDDs and SSDs. Real graphs have a varying density of edges (sparse and dense regions) and these regions can be processed in different ways to achieve superior performance. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 7 / 28
- 8. Graph Organization Graph Organization Each vertex has a set of attributes γ Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 8 / 28
- 9. Graph Organization Graph Organization Vertices are divided into intervals. Intersections between intervals are called blocks Edges are divided into blocks Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 8 / 28
- 10. Graph Organization Graph Organization M-Flash loads in memory some vertex intervals and edges are processed using streaming. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 8 / 28
- 11. Graph Organization Graph Organization Edges create a source-partition (SP) when they are grouped by source interval. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 8 / 28
- 12. Graph Organization Graph Organization Edges create a destination-partition (DP) when they are grouped by destination interval. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 8 / 28
- 13. Graph Organization Number of intervals to process a graph β = φ(T + 1) |V | M M = RAM size |V | = Number of vertices of the graph φ = Size of data per vertex T = Number of threads Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 9 / 28
- 14. Graph Organization Number of intervals to process a graph β = φ(T + 1) |V | M M = RAM size |V | = Number of vertices of the graph φ = Size of data per vertex T = Number of threads For example, 4 bytes of data per node, 2 threads, a graph with 2 billion nodes, and for 1 GB RAM: Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 9 / 28
- 15. Graph Organization Number of intervals to process a graph β = φ(T + 1) |V | M M = RAM size |V | = Number of vertices of the graph φ = Size of data per vertex T = Number of threads For example, 4 bytes of data per node, 2 threads, a graph with 2 billion nodes, and for 1 GB RAM: β = 23 intervals 529 blocks Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 9 / 28
- 16. Processing Model Summary 1 Introduction 2 Graph Organization 3 Processing Model 4 Programming Model 5 Results 6 Conclusions Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 10 / 28
- 17. Processing Model Processing Model Dense Block Processing model DBP Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 11 / 28
- 18. Processing Model Processing Model Dense Block Processing model DBP An efficient model to process blocks with higher density, i.e., blocks with high number of edges. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 11 / 28
- 19. Processing Model Processing Model Streaming Partition Processing Model SPP SP = Source-partition, DP = Destination-partition Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 12 / 28
- 20. Processing Model Processing Model Streaming Partition Processing Model SPP SP = Source-partition, DP = Destination-partition An efficient model for processing sparse blocks. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 12 / 28
- 21. Processing Model Processing Model Bimodal Block Processing Model BBP I/O cost as a metric of efficiency DBP = Dense Block Processing, SSP = Streaming Partition Processing O (DBP (G)) = O (β + 1) |V | + |E| B + β2 O (SPP (G)) = O 2 |V | + |E| + 2 ˆE B + β β = Number of intervals V = Number of Vertices E = Number of Edges ˆE = Number of edges with extended size B = Block size for I/O operations on disk Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 13 / 28
- 22. Processing Model Processing Model Bimodal Block Processing Model BBP I/O cost per block G(p,q) O DBP G(p,q) = O ϑφ (1 + 1/β) + ξψ B O SPP G(p,q) = O 2ϑφ/β + 2ξ(φ + ψ) + ξψ B DBP = Dense Block Processing, SSP = Streaming Partition Processing ξ = Number of edges withing the block G(p,q) ϑ = Number of vertices within the interval φ = Vertex size in bytes ψ = Edge size in bytes B = Block size for I/O operations on disk. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 14 / 28
- 23. Processing Model Processing Model Bimodal Block Processing Model BBP Ratio SPP/DBP O SPP DBP = O 1 β + 2ξ ϑ 1 + ψ φ BlockType G(p,q) = sparse, if O SPP DBP < 1 dense, otherwise DBP = Dense Block Processing, SSP = Streaming Partition Processing Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 15 / 28
- 24. Processing Model Processing Model — Preprocessing 1 Edges are divided in source intervals. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 25. Processing Model Processing Model — Preprocessing 1 The number of edges is measured at the same time to classify the block (sparse or dense). Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 26. Processing Model Processing Model — Preprocessing 2 Source Partitions are rewritten and we store only edges of sparse blocks. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 27. Processing Model Processing Model - an iteration Destination-partitions are generated since the source-partitions. For processing a source-partition, vertex values of the source interval are loaded in memory. The edges are rewritten and divided by destination. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 28. Processing Model Processing Model - an iteration Destination-partitions are generated since the source-partitions. For processing a source-partition, vertex values of the source interval are loaded in memory. The edges are rewritten and divided by destination. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 29. Processing Model Processing Model - an iteration Destination-partitions are generated since the source-partitions. For processing a source-partition, vertex values of the source interval are loaded in memory. The edges are rewritten and divided by destination. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 30. Processing Model Processing Model - an iteration The graph processing starts over the destination-partitions and the dense blocks. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 31. Processing Model Processing Model - an iteration Vertex values of the destination interval are initialized. Next, edges of the destination partition are processed. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 32. Processing Model Processing Model - an iteration Next, dense blocks of the interval are processed. To do the processing, vertex values associated with the source interval are loaded in memory. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 33. Processing Model Processing Model - an iteration Vertex values of the destination interval are initialized. Next, edges of the destination partition are processed. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 34. Processing Model Processing Model - an iteration Next, dense blocks of the interval are processed. To do the processing, vertex values associated with the source interval are loaded in memory. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 35. Processing Model Processing Model - an iteration Vertex values of the destination interval are initialized. Next, edges of the destination partition are processed. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 16 / 28
- 36. Processing Model Programming Model MAlgorithm: Algorithm Interface initialize (Vertex v); gather (Vertex u, Vertex v, EdgeData data); process (Accum v 1, Accum v 2, Accum v out); apply (Vertex v); PageRank degree(v) = out degree for Vertex v; initialize (v): v.value = 0 gather (u, v, data): v.value += u.value / degree(u) process (v 1, v 2, Accum v out): v out = v 1 + v 2 apply (Vertex v): v.value = 0.15 + 0.85 * v.value Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 17 / 28
- 37. Programming Model Summary 1 Introduction 2 Graph Organization 3 Processing Model 4 Programming Model 5 Results 6 Conclusions Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 18 / 28
- 38. Programming Model Programming Model Algorithm for Connected Components initialize (v): v.value = v.id gather (u, v, data): v.value = min (u.value, v.value) process: (v 1, v 2, Accum v out): v out = min (v 1, v 2) Algorithm for Sparse Matrix-Vector Multiplication SpMV initialize (v): v.value = 0 gather (u, v, data): v.value += u.value * data process: (v 1, v 2, Accum v out): v out = v 1 + v 2 Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 19 / 28
- 39. Results Summary 1 Introduction 2 Graph Organization 3 Processing Model 4 Programming Model 5 Results 6 Conclusions Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 20 / 28
- 40. Results Results Datasets Graph Nodes Edges Size LiveJournal 4,847,571 68,993,773 Small Twitter 41,652,230 1,468,365,182 Medium YahooWeb 1,413,511,391 6,636,600,779 Large R-Mat (Synthetic graph) 4,000,000,000 12,000,000,000 Large Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 21 / 28
- 41. Results Results State of the art approaches that are compared with the experiments GraphChi (2012) X-Stream (2013) TurboGraph (2013) MMap (2014) GridGraph (2015) M-Flash Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 22 / 28
- 42. Results Preprocessing time (seconds) LiveJournal Twitter YahooWeb R-Mat GraphChi 23 511 2,781 7,440 X-Stream 5 131 865 2,553 TurboGraph 18 582 4,694 - MMap 17 372 636 - M-Flash 10 206 1,265 4,837 In the worst case, M-Flash is twice slower than the best framework (MMAP) It reads and writes two times the entire graph on disk, which is the third best performance, after MMap and X-Stream. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 23 / 28
- 43. Results Runtime (in seconds) with 8GB of RAM The symbol “-” indicates that the corresponding system failed to process the graph or the information is not available in the respective papers. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 24 / 28
- 44. Results Effect of Memory Size Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 25 / 28
- 45. Conclusions Summary 1 Introduction 2 Graph Organization 3 Processing Model 4 Programming Model 5 Results 6 Conclusions Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 26 / 28
- 46. Conclusions Conclusions M-Flash uses an innovative design that considers the varying density of the graph; By using an adaptive engineering, it benefits from the best of both worlds: stream processing and dense-block processing; Its programming model supports classical and new algorithms straightly adapting to the model of other similar frameworks – among its processing possibilities: PageRank, Connected Components, matrix-vector multiplication, eigensolver, clustering coefficient, to name few; To date, M-Flash is the fastest single-node graph processing framework, beating competitors GraphChi, X-Stream, TurboGraph and MMAP. Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 27 / 28
- 47. Conclusions Acknowledgments CNPq (grant 444985/2014-0) Fapesp (grants 2016/02557-0, 2014/21483-2), Capes NSF (grants IIS-1563816, TWC-1526254, IIS-1217559) GRFP (grant DGE-1148903) Korean (MSIP) agency IITP (grant R0190-15-2012) Thanks Gualdron et al. (1 University of Sao Paulo, Brazil 2 Georgia Institute of Technology, Atlanta, USA 3 Seoul National University, Republic ofSept/2016 28 / 28