![Hierarchical Phasers9/26/105IntroductionShirako & Sarkar in Proc. of IEEE IPDPS 2010 [1]Array accessList accessFirst scalable implementation strategyPredefined tree structureDegree, i.e., tree arityMax. number of tiers, i.e., heightComposed from phasersProblematicNone dynamic structureTwo-phase support incompleteLeaves design decisions open PhaserTier 0subsubTier 1subsubsubsubTier 2(leafs)sigsigsigsigsigsigsigsigA1A2A3A4A5A6A7A8](https://image.slidesharecdn.com/hpcc2010-flattend-100926085424-phpapp02/85/Insertion-Tree-Phasers-Efficient-and-Scalable-Barrier-Synchronization-for-Fine-grained-Parallelism-5-320.jpg)
Insertion Tree Phasers: Efficient and Scalable Barrier Synchronization for Fine-grained Parallelism
- 1. Insertion Tree PhasersEfficient and Scalable Barrier Synchronization for Fine-grained ParallelismStefan MarrS. Verhaegen, B. De Fraine, T. D’Hondt, W. De MeuterSoftware Languages LabVrijeUniversiteitBrussel
- 2. AgendaIntroductionBarriers, PhasersInsertion Tree PhasersInsertion TreePhaser AlgorithmEvaluationSummary9/26/102
- 3. BarriersSynchronizing parallel activitiesHigh productivity: easy to get right Mostly for scientific computingMany-core evolutionSynchronizing dynamic and irregular problemsRequires low-overhead dynamic hierarchical barriers9/26/103Introductiont1p1t2p1t3p1t1p2t2p2t3p2t1p3t2p3t3p3
- 4. t1p1Phasers9/26/104IntroductionExtension of X10 clocksClocks: dynamic two-phase barrier for fork/join parallelismRegistration modes for barrierEnables expression of producer/consumer relationSingle statementsExecuted only by single thread, avoids duplicated barrier operationst1p2t2p2t3p2t2p2t3p2t2p3t3p3
- 5. Hierarchical Phasers9/26/105IntroductionShirako & Sarkar in Proc. of IEEE IPDPS 2010 [1]Array accessList accessFirst scalable implementation strategyPredefined tree structureDegree, i.e., tree arityMax. number of tiers, i.e., heightComposed from phasersProblematicNone dynamic structureTwo-phase support incompleteLeaves design decisions open PhaserTier 0subsubTier 1subsubsubsubTier 2(leafs)sigsigsigsigsigsigsigsigA1A2A3A4A5A6A7A8
- 6. Open Questions withHierarchical PhasersDynamic tree construction, or on initialization?Tradeoffs for atomic operations, overhead of joining/leaving phaserHow are operations synchronized?Tradeoffs for overheads and restrictions on parallelismGarbage collection problem for dropped participantsKeeps list of synchronization objects incl. dropped participantsAfter reaching max. #participantsIs the tree rebalanced? (Hint at it for dropped nodes)Two-phase barrier support does not hide latency for original phasers9/26/106Introduction
- 8. Design GoalSupport for full generality of Phaser propertiesTwo-phase supportSignal-only/wait-only for producers/consumersSingle statementFull dynamicity: fine-grained hierarchical fork/joinAdaptation of existing, scalable approachesDissemination barrier not adaptableRemaining are tree-based approaches9/26/108Insertion TreePhaserAlgorithm
- 9. Insertion TreeGoalsStable, i.e., minimized tree modificationsAvoid inconsistent synchronization informationMaximizing parallel operationsSolution: Insertion TreeInverted treeNo removalComplete smallest subtree first9/26/109Insertion TreePhaserAlgorithm1/2
- 17. Determining the Insertion PointdefgetNextInsertNode(tree): result = tree.lastNodei = tree.numLeaveswhileimod 2 == 0: result = result.parenti = i/2return result # this is for 2-ary trees # is adaptable for n-ary trees, too9/26/1017Insertion TreePhaserAlgorithm
- 18. Synchronization Tree*9/26/1018Insertion TreePhaserAlgorithmPhaserphase: 000Phase counter0000woHelper nodesWait-only flagPhase counter0000rsmdParticipant nodesResume flag*) is simplified, leaves out registration modesA1A2A3A4
- 19. Announcing Synchronization9/26/1019Insertion Tree Phaser AlgorithmPhaserphase: 00000000000A1A2A3A4
- 20. Announcing Synchronization9/26/1020Insertion Tree Phaser AlgorithmPhaserphase: 0000110001rsmd1rsmdA1A2A3A4
- 21. Announcing Synchronization9/26/1021Insertion Tree Phaser AlgorithmPhaserphase: 00011111rsmd1rsmd1rsmd1rsmdA1A2A3A4
- 22. Announcing Synchronization9/26/1022Insertion Tree Phaser AlgorithmPhaserphase: 00111111rsmd1rsmd1rsmd1rsmdA1A2A3A4
- 23. Announcing Synchronization9/26/1023Insertion Tree Phaser AlgorithmPhaserphase: 01111111rsmd1rsmd1rsmd1rsmdA1A2A3A4
- 24. Announcing Synchronization9/26/1024Insertion Tree Phaser AlgorithmSynchronization reached.Continue to next phase.Phaserphase: 11111111rsmd1rsmd1rsmd1rsmdA1A2A3A4
- 25. Dropping Participants9/26/1025Insertion TreePhaserAlgorithmPhaserphase: 0010011001rsmd1rsmdA1A2A3A4
- 26. Dropping Participants9/26/1026Insertion TreePhaserAlgorithmPhaserphase: 0010wo1101rsmd1rsmdA1A2A3A4
- 27. h1:RDropping Participants9/26/1027Insertion TreePhaserAlgorithmPhaserphase: 001wowo111rsmd1rsmdA1A2A3A4
- 28. h1:RDropping Participants9/26/1028Insertion TreePhaserAlgorithmPhaserphase: 0wo1wowo111rsmd1rsmdA1A2A3A4
- 29. Dropping Participants9/26/1029Insertion TreePhaserAlgorithmSynchronization reached.Continue to next phase.Phaserphase: 1h1:Rwo1wowo111rsmd1rsmdA1A2A3A4
- 30. h1:RDropping Participants9/26/1030Insertion TreePhaserAlgorithmPhaserphase: 1wo1h1:Lwowo111rsmd1rsmdA1A2A3A4
- 31. Adding New Participants9/26/1031Insertion TreePhaserAlgorithmPhaserphase: 898899rsmd89rsmd8A1A2A3A4
- 32. Adding New Participants9/26/1032Insertion TreePhaserAlgorithmPhaserphase: 89888899rsmd89rsmd8A1A2A3A4
- 33. Adding New Participants9/26/1033Insertion TreePhaserAlgorithmPhaserphase: 8-188+198899rsmd89rsmd8A1A2A3A4
- 34. Adding New Participants9/26/1034Insertion TreePhaserAlgorithmPhaserphase: 888propagate phase count98899rsmd89rsmd8A1A2A3A4
- 37. Overhead: Two-Phase vs. Classic9/26/1037Evaluation
- 38. Use as Drop-In Replacement for SPLASH-2Speedup compared to TmcSpinBarrier9/26/1038Evaluation
- 39. SummaryScalable and efficient approach to PhasersDocuments implementationBased on fully dynamic insertion treeOvercomes limitations of existing approachesUsable as drop-in replacementFuture workScalability beyond 59 coresOptimization for other memory architectures9/26/1039Stefan Marr, IEEE HPCC 2010, Insertion TreePhasers
- 40. 9/26/1040Stefan Marr, IEEE HPCC 2010, Insertion TreePhasersQuestions?Phaserphase: 1h1:Rwo1h1:Lwowo11Implementationhttp://barriers.googlecode.com/MIT license1rsmd1rsmdA1A2A3A4
- 41. References[1] Shirako, Jun & Sarkar, Vivek: Hierarchical Phasers for Scalable Synchronization and Reductions in Dynamic Parallelism In: Proc. of IEEE IPDPS (2010).9/26/1041
Editor's Notes
- #5: Shirako et al.X10 Vijay Saraswat
- #6: Shirako + Sarkar
- #8: So I went to the whiteboard drew a tree and figured out how to do it slightly different
- #10: How to build a tree to synchronize dynamic parallelism?
- #19: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #20: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #21: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #22: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #23: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #24: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #25: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #26: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #27: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #28: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #29: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #30: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #31: Example tree like in paper, briefly the different properties, and that they are aggregations of the subtree
- #32: In the general case: - propagate the phase count minimum up the tree - while doing this, wait for racing values, by checking that the found value is the expected from the last visited node, if it is not, wait until it is, thus the racing activity passed
- #33: In the general case: - propagate the phase count minimum up the tree - while doing this, wait for racing values, by checking that the found value is the expected from the last visited node, if it is not, wait until it is, thus the racing activity passed
- #34: In the general case: - propagate the phase count minimum up the tree - while doing this, wait for racing values, by checking that the found value is the expected from the last visited node, if it is not, wait until it is, thus the racing activity passed
- #35: In the general case: - propagate the phase count minimum up the tree - while doing this, wait for racing values, by checking that the found value is the expected from the last visited node, if it is not, wait until it is, thus the racing activity passed