![CLASSIFICATION OF PERFORMANCE
VISUALIZATION TECHNIQUES
Category Performance Visualization Example applications and studies
Techniques
Simple visual Pie charts, distribution, box plots, ParaGraph [2], PET [20], SvPablo [16],
structures kiviat diagrams VAMPIR [21], Devise [22], AIMS [9]
Timeline views Paje [23], AIMS [9], Devise [22],
AerialVision [24], Paraver [25],
SIEVE [14], Virtue [13], utilization and
algorithm timeline views in [17]
Information typologies SHMAP [26], Vista [4], Voyeur [27],
processor and network port display in
[28], hierarchical display in [12]
Information landscape Triva [29], Cichild [30]
Trees & networks Paradyn [18], Cone Trees [31],
Virtue [13], [32]
Composed visual Single-axis composition AIMS [9], Vista [4]
structures Double-axis composition Devise [22], AerialVision [24]
Case composition Triva [29]
Interactive visual Interaction through controls (data Paje[23], data input, filtering,
structure input, data transformation, visual and view manipulation in [28]
mapping definition, view operations) and [32]
Interaction through images Virtue [13], Cone Trees [31],
(magnifying lens, cascading displays, Devise [22], direct manipulation of the
linking and brushing, direct 3D cone and virtual threads in [32]
manipulation of views and objects)
Focus + context Macro-micro composite view Microscopic profile in [4],
visual structures PC-Histogram in [24]
Performance Visualization for Large-scale Computing Systems: A Literature Review 10](https://image.slidesharecdn.com/performancevisualization-13105942780902-phpapp01-110713170145-phpapp01/85/HCII2011-Performance-Visualization-for-Large-Scale-Computing-System-A-Literature-Review-10-320.jpg)
![SIMPLE VISUAL STRUCTURES
• Statistical charts
• Provide an overview of
important performance
metrics
• Enable quick identification a. PET: Bar chart of resource utilization b. Pajé Pie chart representing the percentage
:
of major problems
percentage of different processors [22] of time with different number of active
threads at a node [17]
c. SvPablo: color matrix of metrics, each d. ParaGraph: Kiviat diagram showing load
column representing a performance metric, imbalance among different processors [7]
and color representing the value [13]
Performance Visualization for Large-scale Computing Systems: A Literature Review 11](https://image.slidesharecdn.com/performancevisualization-13105942780902-phpapp01-110713170145-phpapp01/85/HCII2011-Performance-Visualization-for-Large-Scale-Computing-System-A-Literature-Review-11-320.jpg)
![SIMPLE VISUAL STRUCTURES
• Trees and networks
a. Paradyn: Performance Consultant, b. Cone Trees: 3D visualization of tree
showing a search hierarchy [14] structures [31]
Virtue: Geographic network display [15]
Performance Visualization for Large-scale Computing Systems: A Literature Review 17](https://image.slidesharecdn.com/performancevisualization-13105942780902-phpapp01-110713170145-phpapp01/85/HCII2011-Performance-Visualization-for-Large-Scale-Computing-System-A-Literature-Review-17-320.jpg)
[HCII2011] Performance Visualization for Large Scale Computing System - A Literature Review
- 1. Q I N G A O 1, X U H U I Z H A N G 1, P E I - L U E N P A T R I C K R A U 1 1 INSTITUTE OF HUMAN FACTORS & ERGONOMICS, DEPT. OF INDUSTRIAL ENGINEERING, TSINGHUA UNIVERSITY, BEIJING, 100084, CHINA A N T H O N Y A . M A C I E J E W S K I 2, H O W A R D J A Y S I E G E L 2,3 2E L E C T R I C A L A N D C O M P U T E R E N G I N E E R I N G D E P A R T M E N T , 3C O M P U T E R S C I E N C E D E P A R T M E N T COLORADO STATE UNIVERSITY, FORT COLLINS, CO 80523 -1373 USA PERFORMANCE VISUALIZATION FOR LARGE-SCALE COMPUTING SYSTEMS A Literature Review HCI International 2011 9-14 July, Orlando, USA
- 2. CONTENT • Motivation • Approach to Performance Visualization • Review of Performance Visualization Techniques for Large-Scale Systems • Future Work Performance Visualization for Large-scale Computing Systems: A Literature Review 2
- 3. MOTIVATION Exascale computers: 1000 times faster than the current Need for extreme scale petascale systems computing solutions Immense volume and Need to performance complexity of the monitoring & tuning in run- performance data time for extreme-scale systems Need for powerful and A review of existing usable performance performance visualization methods visualization tool for extreme- and tools for large scale system scale systems Performance Visualization for Large-scale Computing Systems: A Literature Review 3
- 4. PERFORMANCE VISUALIZATION Program Visualization Visual behavior Representations Data View Visual transformation Transformation Mappings Raw Data Views data tables Source: Card, 2002 Human Interaction • Goal: • Augmenting cognition with the human visual system’s highly tuned ability to see patterns and trends • Aid comprehension of the dynamics, intricacies, and properties of program execution Performance Visualization for Large-scale Computing Systems: A Literature Review 4
- 5. APPROACH TO PERFORMANCE VISUALIZATION Enabling access to performance data to be Instrumentation measured Recording selected data during the run-time of the Measurement program Data analysis Analyzing data for performance visualization Mapping performance characteristics to proper Visualization visual representations and interactions Performance Visualization for Large-scale Computing Systems: A Literature Review 5
- 6. APPROACH TO PERFORMANCE VISUALIZATION • Instrumentation • What to be instrumented? Fidelity Reflect application Minimizing performance as perturbation of Pertubation closely as possible that behavior as much as possible • Approach • Hardware • Less performance degradation • Poor portability • Software • Better portability • Automation required for large-scale systems Performance Visualization for Large-scale Computing Systems: A Literature Review 6
- 7. APPROACH TO PERFORMANCE VISUALIZATION • Measurement • Tracing • More detailed execution information • Necessary for visualizing detailed program run-time behaviors • E.g., Virtue, Pajé • Profiling • Collects only summary statistics, mostly with hardware counters • Less pertubation by sacrificing fidelity • Allow data collection with long execution time • E.g., SvPablo • Trigger for recording action • Event-driven • Periodically (sampling) • Real-time or post-mortem? • For distributed application, real-time measurement and visualization is necessary Performance Visualization for Large-scale Computing Systems: A Literature Review 7
- 8. APPROACH TO PERFORMANCE VISUALIZATION • Data analysis • Microscopic and macroscopic metrics • Method • Data reduction • Multivariate statistical analysis • Application-specific analysis • Bates, 1995: Recognizing high-level program behaviors • AIMS: Pointing out causes of poor performance, generating scalability trends Performance Visualization for Large-scale Computing Systems: A Literature Review 8
- 9. APPROACH TO PERFORMANCE VISUALIZATION • Visualization • Basic visual components involved in information visualization (Card, 2002) • Spatial substrate • Marks • Connections • Enclosures Types of marks, source: Card, 2002 • Retinal properties • Temporal encoding Retinal properties, source: Card, 2002 Performance Visualization for Large-scale Computing Systems: A Literature Review 9
- 10. CLASSIFICATION OF PERFORMANCE VISUALIZATION TECHNIQUES Category Performance Visualization Example applications and studies Techniques Simple visual Pie charts, distribution, box plots, ParaGraph [2], PET [20], SvPablo [16], structures kiviat diagrams VAMPIR [21], Devise [22], AIMS [9] Timeline views Paje [23], AIMS [9], Devise [22], AerialVision [24], Paraver [25], SIEVE [14], Virtue [13], utilization and algorithm timeline views in [17] Information typologies SHMAP [26], Vista [4], Voyeur [27], processor and network port display in [28], hierarchical display in [12] Information landscape Triva [29], Cichild [30] Trees & networks Paradyn [18], Cone Trees [31], Virtue [13], [32] Composed visual Single-axis composition AIMS [9], Vista [4] structures Double-axis composition Devise [22], AerialVision [24] Case composition Triva [29] Interactive visual Interaction through controls (data Paje[23], data input, filtering, structure input, data transformation, visual and view manipulation in [28] mapping definition, view operations) and [32] Interaction through images Virtue [13], Cone Trees [31], (magnifying lens, cascading displays, Devise [22], direct manipulation of the linking and brushing, direct 3D cone and virtual threads in [32] manipulation of views and objects) Focus + context Macro-micro composite view Microscopic profile in [4], visual structures PC-Histogram in [24] Performance Visualization for Large-scale Computing Systems: A Literature Review 10
- 11. SIMPLE VISUAL STRUCTURES • Statistical charts • Provide an overview of important performance metrics • Enable quick identification a. PET: Bar chart of resource utilization b. Pajé Pie chart representing the percentage : of major problems percentage of different processors [22] of time with different number of active threads at a node [17] c. SvPablo: color matrix of metrics, each d. ParaGraph: Kiviat diagram showing load column representing a performance metric, imbalance among different processors [7] and color representing the value [13] Performance Visualization for Large-scale Computing Systems: A Literature Review 11
- 12. SIMPLE VISUAL STRUCTURES • Time-line views • Showing the evolution of performance statistics over time Utilization and overhead view in Alexandrov et al., 2010 Time views of utilization/computation/communication metrics of AerialVision AerialVision’s time view of runtime warp divergence breakdown Performance Visualization for Large-scale Computing Systems: A Literature Review 12
- 13. SIMPLE VISUAL STRUCTURES • Time-line views • Describing run-time behaviors and communication paths Virtue: time-tunnel display Pajé visualization of program execution and communication : AIMS: visualization of program executions ParaGraph: Space-time diagram Performance Visualization for Large-scale Computing Systems: A Literature Review 13
- 14. SIMPLE VISUAL STRUCTURES • Time-line views • • Facilitating source code level analysis AerialVision: PC-Histogram SIEVE: Contour-plot showing calls to a specific function Performance Visualization for Large-scale Computing Systems: A Literature Review 14
- 15. SIMPLE VISUAL STRUCTURES • Information typography Proposed hierarchical views of a complex reconfigurable Port display showing job computing application allocation, communication traffic, and route between nodes of a cluster Performance Visualization for Large-scale Computing Systems: A Literature Review 15
- 16. SIMPLE VISUAL STRUCTURES • Information landscape a. Triva: information landscape based on b. Triva: information landscape based network typology on resource hierarchy c. Cichild: interpolated surfaces showing network delays between different sites Performance Visualization for Large-scale Computing Systems: A Literature Review 16
- 17. SIMPLE VISUAL STRUCTURES • Trees and networks a. Paradyn: Performance Consultant, b. Cone Trees: 3D visualization of tree showing a search hierarchy [14] structures [31] Virtue: Geographic network display [15] Performance Visualization for Large-scale Computing Systems: A Literature Review 17
- 18. COMPOSED STRUCTURE • Single-axis composition • Multiple graphs sharing single axis • Double-axis composition • Multiple graphs sharing AIMS: composite view of procedure execution graph on double axis each node and machine-load chart of each node • Case compositions • Two graphs having a single mark for each case fused Devise: message behavior visualization Performance Visualization for Large-scale Computing Systems: A Literature Review 18
- 19. INTERACTIVE STRUCTURES • Direct interaction through the visualization • Magifying lens • Panning, selecting, re-positioning • Cascading display (e.g., ConeTrees) • Use of gestures (e.g., Virtue) • Indirect interaction through controls • Interactions with underlying computation,Virtue: Magnifying lens such as data-related controls and definitions of visual mapping • View configurations • Scroll-bars, zoom in/out, sliders… Performance Visualization for Large-scale Computing Systems: A Literature Review 19
- 20. ATTENTION-REACTIVE VISUAL STRUCTURES • Limited usage in performance visualization systems AerialVision: PC histogram Vista: Filmstrip view of utilization Performance Visualization for Large-scale Computing Systems: A Literature Review 20
- 21. SUMMARY & OUTLOOK • Summary issues that need to be addressed throughout the process of performance visualization • Review performance visualization techniques from 21 systems • Challenge: huge data size requires good scalability • Data abstraction method from scientific visualization • Visualization based on focus + context abstraction • Challenge: ergonomics and usability issues • Understanding of characteristics and limitations and human sensory and cognition capabilities Performance Visualization for Large-scale Computing Systems: A Literature Review 21
- 22. THANKS, AND QUESTIONS? Performance Visualization for Large-scale Computing Systems: A Literature Review 22