A Structured Approach for Conducting a Series of Controlled Experiments in Software Visualization [IVAPP 2014]
0 likes112 views
The document presents a structured approach for conducting controlled experiments in software visualization, emphasizing the need for empirical research in the field. It discusses various influencing factors such as user roles, software artifact types, and visualization techniques, alongside an example experiment comparing 2D and 3D visualizations. The approach is based on Munzner's model and aims to facilitate better planning and assessment of research in software visualization.
![4
Institut für Wirtschaftsinformatik/Informatik
Munzner‘s Model and its Extension
© 2014 by R. Müller
[Munzner 2009; Meyer, Sedlmair, Munzner 2012]
Block Guideline](https://image.slidesharecdn.com/ivapp2014-structuredapproachforcontrolledexperiments-181002090155/85/A-Structured-Approach-for-Conducting-a-Series-of-Controlled-Experiments-in-Software-Visualization-IVAPP-2014-4-320.jpg)
A Structured Approach for Conducting a Series of Controlled Experiments in Software Visualization [IVAPP 2014]
- 1. Institut für Wirtschaftsinformatik/Informatik 5th International Conference on Information Visualization Theory and Applications Richard Müller1, Pascal Kovacs1, Jan Schilbach1, Ulrich Eisenecker1, Dirk Zeckzer2, Gerik Scheuermann2 1Information Systems Institute 2Institute of Computer Science University of Leipzig, Leipzig, Germany January 5, 2014 A Structured Approach for Conducting a Series of Controlled Experiments in Software Visualization
- 2. 2 Institut für Wirtschaftsinformatik/Informatik 2D or 3D: State of the Art © 2014 by R. Müller Scientific drivers Lack of empirical research in software visualization Ongoing discourse: 2D vs. 3D Five papers about 3D software visualizations at VISSOFT 2013 Technical drivers Innovative methods to generate software visualizations Increasing computing power Emerging technology High quality and low cost 3D environments 3D-Vision Glasses and Infrared Transmitter 3D-Vision Capable Monitor or Projector Quadro GPU
- 3. 3 Institut für Wirtschaftsinformatik/Informatik A Series of Controlled Experiments © 2014 by R. Müller A series of controlled experiments is required to determine the circumstances when and why a specific software visualization is suitable for a certain software engineering task Software Visualization User Task Software Artifact Represen- tation Navigation & Interaction Implemen- tation
- 4. 4 Institut für Wirtschaftsinformatik/Informatik Munzner‘s Model and its Extension © 2014 by R. Müller [Munzner 2009; Meyer, Sedlmair, Munzner 2012] Block Guideline
- 5. 5 Institut für Wirtschaftsinformatik/Informatik Domain Specific Adaption of Munzner’s Extended Model for Software Visualization © 2014 by R. Müller
- 6. 6 Institut für Wirtschaftsinformatik/Informatik Possible Instantiations (1/3) Factor/Sub- Factor Examples for Possible Instantiations User Role Background Knowledge Circumstances Manager, Requirements Engineer, Architect, Developer, Tester, Maintainer, Reengineer, Documenter, Consultant, Team, Researcher Age, Gender, Color Blindness, Ability of Stereoscopic Viewing Education, Programming Experience, Domain Knowledge Occupation, Familiarity with Study Object/Tools Task Problem Development, Maintenance, Re-Engineering, Reverse Engineering, Software Process Management, Marketing, Test, Documentation Operation Retrieve Value, Filter, Compute Derived Value, Find Extremum, Sort, Determine Range, Characterize Distribution, Find Anomalies, Cluster, Correlate © 2014 by R. Müller
- 7. 7 Institut für Wirtschaftsinformatik/Informatik Possible Instantiations (2/3) Factor/Sub- Factor Examples for Possible Instantiations Software Artifact Type Size Aspect Requirements, Architecture, Source Code, Stack Trace, Revision History Small, Medium, Large Structure, Behavior, Evolution Representation Dimensionality Technique 2D, 2.5D, Augmented 2D, Adapted 2D, Inherent 3D Graph, Tree, Abstract/Real World Metaphor, Decorational/Representational Animation © 2014 by R. Müller
- 8. 8 Institut für Wirtschaftsinformatik/Informatik Possible Instantiations (3/3) Factor/Sub- Factor Examples for Possible Instantiations Navigation & Interaction Technique Input Output Overview, Zoom, Filter, Details-on-Demand, Relate, History, Extract Keyboard, Mouse, Gamepad, Flystick, Kinect, Touch Device, Leap Motion, Brain-Computer Interface Paper, Monitor, Projector, Virtual Reality Environment, Oculus Rift Implementation Algorithm Platform Dep. Automation Radial Layout, Balloon Layout, Treemap, Information Cube, Cone Tree Platform Platform Independent, Platform Dependent Full, Semi, Manual Data Famix, Dynamix, Hismo © 2014 by R. Müller
- 9. 9 Institut für Wirtschaftsinformatik/Informatik Example Experiment: Overview Research question Does an inherent 3D software visualization reduce time to solve software engineering tasks, compared to a 2D software visualization? Dependent variable time Independent variable dimensionality Between-subjects design control group (2D), experimental group (3D) © 2014 by R. Müller
- 10. 10 Institut für Wirtschaftsinformatik/Informatik Example Experiment: Detail © 2014 by R. Müller User Role Background Knowledge Developer Color Blindness, Ability of Stereoscopic Viewing Education, Programming Experience, Domain Knowledge (Virtual Reality, Touch Devices, 3D) Task Problem Find a Bug, Identify a Dominating Class Operation Retrieve Value, Filter, Find Extremum, Find Anomalies, Correlate Software Artifact Type Size Aspect Source Code Medium Structure Representation Dimensionality Technique 2D vs. Inherent 3D Graph (Nested Node-Link Technique) Navigation & Interaction Technique Input Output Overview, Zoom, Filter, Relate Touch Device Virtual Reality Environment Implementation Algorithm Platform Dep. Automation Force Directed Layout Plaform Independent Full Data Abstraction Famix Vary different factors in different experiments while keeping other relevant factors constant or measure their influence on the result
- 11. 11 Institut für Wirtschaftsinformatik/Informatik Summary and Conclusion The structured approach to conduct a series of controlled experiments in software visualization is based on Munzner‘s model and its extension The approach… Provides a comprehensive overview of relevant influence factors and their possible interactions Helps to plan new experiments, to categorize previous experiments and to assess their results Supports researchers to identify research gaps © 2014 by R. Müller
- 12. 12 Institut für Wirtschaftsinformatik/Informatik Your questions… © 2014 by R. Müller
- 13. 13 Institut für Wirtschaftsinformatik/Informatik References Meyer, M., Sedlmair, M. & Munzner, T., 2012. The four-level nested model revisited: blocks and guidelines. In Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization (BELIV). pp. 1–6. Munzner, T., 2009. A nested model for visualization design and validation. IEEE Transactions on Visualization and Computer Graphics, 15(6), pp.921–928. © 2014 by R. Müller