Heuristic Evaluation of Immersive 3D Application
Download as DOC, PDF1 like224 views
The document summarizes an evaluation of a visualization application using Shneiderman's visualization mantra and additional heuristics. It was difficult for users to make inferences from the dense data. The evaluation tested the application's ability to provide overview, zooming and filtering, details on demand, and linking of selections. Suggestions included providing more context, statistics, and allowing saving of views and selections to link between multiple coordinated views. Limiting shape and texture options and improving color differentiation was also suggested.
Heuristic Evaluation of Immersive 3D Application
- 1. Visualization Evaluation Utilizing the Shneiderman Mantra Matthew C. Doyle State University of New York at Oswego State University of New York at Oswego – 7060 New York 104 – Oswego, New York 1 STATE UNIVERSITY OF NEW YORK AT OSWEGO | HUMAN – COMPUTER INTERACTION
- 2. INTRODUCTION The visualization walkthrough highlighted possible deficiencies in the applications capacity to deal with the issue of information density. While the walkthrough of the interface confirmed that users can access the information associated with the given data points, it was still difficult to make inferences towards possible relationships and trends held within the data. As a result, a heuristic evaluation was conducted to focus strictly on the challenges facing users that seek to visualize new relationships, patterns and correlations in large synoptic sky surveys. EVALUATION MECHANISMS In an effort to evaluate information visualizations, one must take into account both the usability issues of the application as well as the expressiveness and quality of the visual representation (Freitas et al, 2002). Expressiveness and quality can be determined by how effective the visualization is in giving expert users the ability to gain insight about the data they are investigating. This ability comes from giving users control over the density of both the dataset and the visualization. Drill- down methodology can be implemented to effectively give users the opportunity to determine the context in which reduction or refinement should occur. The following evaluation mechanisms were chosen to gain insight into the applications efficacy in allowing users to gain insight from the visualizations they produced. APPLYING SHNEIDERMANS MANTRA To establish a practical approach for evaluating information visualizations, Ben Shneiderman established the Visual Information-Seeking Mantra (1996) to describe the functionality thata visualization technique should provide (Shneiderman, 1996). The basic principles of this methodology consist of providing the user with an overview of the data, allowing the user to selectively zoom and filter relevant information, bring up details about the dataset based on a specified data points, and highlight a specific subset data points and view them separately through different visualizations (Craft & Cairns, 2005). Together, these ideals are designed to allow the users to drill into a large dataset and retrieve bits of interesting information to be compared on a smaller scale. OVERVIEW Description: In the overview, the user should be able to identify interesting patterns and focus on one multiple more closely. Significant features can be isolated and selected for further examination, aiding the user in filtering extraneous information so that they can complete their task more efficiently by excluding unimportant aspects of the representation (Stephens, 2003). Analysis: The iViz application currently allows the user to browse through a large dataset using a 3D panoramic viewing perspective. Users can also filter extraneous information by zooming in on a particular subset of data, unchecking shape and State University of New York at Oswego – 7060 New York 104 – Oswego, New York 2 STATE UNIVERSITY OF NEW YORK AT OSWEGO | HUMAN – COMPUTER INTERACTION
- 3. texture parameters, and using the each function at the bottom of the screen. Suggestions: Users may want to know some overlying statistics about the dataset as they browse so they have a better idea of the data and can detect outliers quicker. ZOOM & FILTER Description: Zooming and filtering are also crucial techniques that can be used to overcome information density. Zooming has two functions; to display the data objects larger and to present additional details about the data as it zooms in. Filtering allows the user to hide or reveal data of interest so the information can be simplified to aid cognition. Dynamic filtering allows users to quickly see how the changed variable affects the data visualization with search filters. Dynamic queries allow the user to adjust the parameters of a database query in order to return results by keyword, category, range, date, etc. (Craft & Cairns, 2005). Analysis: Zooming and filtering techniques are both utilized in the iViz application. Users can zoom throughout the dataset in 3D and filter information by checking or unchecking shape or texture features. There is also a search function that allows the users to filter out certain data points that don't fit a range specified by the user. Suggestions: The zooming affordance works well but it would be helpful to allow the users to hit a button where they can return to their original position when zooming or save their current perspective. Zooming could also bring in additional information about the data points, such as descriptive statistics. The system could make better use of dynamic filtering by allowing the users to filter by a range of values, keywords, categories, or dates. DETAILS ON DEMAND Description: Details on demand allow users interactively select parts of data to be visualized more detailed while providing an overview of the whole informational concept. This technique also provides supplementary information on a point-by- point basis without requiring a change of view. This is useful for relating the detailed information to the rest of the data set or for quickly solving particular tasks, such as identifying a specific data element amongst many, or relating attributes of two or more data points. Analysis: The iViz application provides these details when a user selects a specific data point. From here, a window emerges with all of the variable information associated with the data point. This allows users to uncover new information without changing the representational context in which the data is arranged. Suggestions: The application would benefit from providing descriptive statistics and context towards the variables that may hold outliers. The details on demand feature should allow users to begin to discover data points they might want to highlight and visualize further. State University of New York at Oswego – 7060 New York 104 – Oswego, New York 3 STATE UNIVERSITY OF NEW YORK AT OSWEGO | HUMAN – COMPUTER INTERACTION
- 4. LINKING & BRUSHING Description: Connecting multiple visualizations through interactive linking and brushing provides more information than considering the larger visualization independently (Keim, 2002). As a result, the idea of linking and brushing is to combine different visualization methods to overcome the shortcomings of single techniques. Analysis: The application currently does not allow the user to highlight specific data points and visualize them separately from the entire dataset. Suggestions: Many users insisted that they would like to see a feature that allows them to select a group of data points and visualize them in different ways. It would be helpful to allow users to save subsets of data points while browsing through the dataset and apply them to linked visualizations to aid in the discovery of dependencies and correlations. ADDITIONAL EVALUATION CRITERIA In “Evaluating usability of information visualization techniques” Frietas et al (2002) proposed three additional evaluation parameters to measure the effectiveness of a visualization space. These metrics include completeness, spatial organization, codification of information, and state transition. Alongside the Shneiderman Mantra, these heuristics serve as a suitable foundation for visualization evaluation. COMPLETENESS Description: The concept of representing all the semantic contents of the data to be displayed. This is affected by the geometric or visual constraints (size of the display, maximum number of data elements, etc.) imposed by the visual representation as well as by its cognitive complexity, which in turn can be measured by data density, data dimension and by the relevance of the displayed information (Freitas et al, 2002). Analysis: The current iteration of this system suffers from information overload in that it overwhelms the user with a lot of information with limited viewing perspectives and no affordances for selecting specific data points and highlighting them to view them from multiple viewpoints and different visual representations. Suggestions: Users should be able to extract important findings and save their work to avoid unnecessary redundancy. Allow extraction of sub-collections and of query parameters. SPATIAL ORGANIZATION State University of New York at Oswego – 7060 New York 104 – Oswego, New York 4 STATE UNIVERSITY OF NEW YORK AT OSWEGO | HUMAN – COMPUTER INTERACTION
- 5. Description: Related to the overall layout of a visual representation, which comprises analyzing how easy is to locatean information element on the display and to be aware of theoverall distribution of information elements in the representation. The spatial orientation, which contributes for the user being aware of the distribution of information elements, is dependent on the presentation of context while displaying a specific element in detail (Freitas et al, 2002). Analysis: Some data points overlap each other, making it difficult to differentiate some points from others. The shape and texture elements are sometimes troubling for users because they cannot differentiate between the two classifiers. Utilization of poor color also makes some data points hard to see. Suggestions: Users would benefit from the simplification of shape and texture parameters that are available for variable assignment. Many users had trouble distinguishing between similar shapes and textures. The application should use three to four different shapes and no more than two textures. CODIFICATION OF INFORMATION Description: The use of additional symbols or realistic characteristics can be used either for building alternative representations (like groups of elements in clustered representations) or to aid in the perception of information elements (Freitas et al, 2002). Analysis: The application currently utilizes location (XYZ), color (RGB), shape, texture, and opacity to codify variables into visual representations. Users have expressed difficulty distinguishing between elements due to poor use of color and texture. Suggestions: Although variables can be given many different visual attributes, the data can still overwhelm the user. At times users struggled to differentiate between shapes and textures, which led them to select the wrong class when attempting to retrieve information from a specific star. iViz should primarily rely on color to group clusters of information together and visualize them in small multiples, using fewer shapes and textures as classifiers. STATE TRANSITION Description: The result of rebuilding the visual representation after a user action. The time spent by the technique to do that and the changes in spatial organization of the resulting image are important factors that can affect the perception of State University of New York at Oswego – 7060 New York 104 – Oswego, New York 5 STATE UNIVERSITY OF NEW YORK AT OSWEGO | HUMAN – COMPUTER INTERACTION
- 6. information (Freitas et al, 2002). Analysis: Currently, the application allows users to change state by zooming in and out of a large 3D data set represented as a bubble chart, but there is no mechanism for users to revert back to their original position or to a specific view. As a result, it can be difficult to recreate exact visualizations. Suggestions: Users would benefit from the ability to revert to a familiar spot in case they get lost or to go to a specified position in the 3D visualization to recreate exact perspectives. The ability to take screenshots of a perspective in the 3D representation or of a specific visualization could also help users share information with collaborators or save it for future investigation. OVERVIEW OF SUGGESTIONS Overview is given but participants of the walkthrough oftentimes got confused while exploring in 3D. Many users wanted the opportunity undo or redo function to revert back to a previous view of the data. Zooming functionality allowed users to identify a specific data point that adheres to specific constraints. Further, the capability to generate details on demand from clicking on a specific data point was efficiently integrated into the application. However, users were not able to highlight interesting data points and link them together to create lists. It would be helpful for expert users to gain insight towards possible relationships within the data by allowing them to explore the dataset in an immersive 3D representation while being able to receive details on demand on specific data points and save ones that are of interest. Users could then link these data points together, and visualize the across multiple small visualizations. This would allow the user to drill down into the data and break it down into small linked fragments of information. VISUALIZATION Users suggested adding context to what certain variables represented in the data set, as well as the ability to preview the visualization before choosing to map a certain variable to a given attribute. Lastly, users also requested the ability to compare two different sets of visualizations at once. This ability could be implemented by a mechanism that provides small multiples of information for the user. Small Multiples Following the proposed interface functionality of filtering, brushing, and linking, small multiples would be helpful to aid users in reducing information density by using the linked data points to generate quick comparisons amongst each other. State University of New York at Oswego – 7060 New York 104 – Oswego, New York 6 STATE UNIVERSITY OF NEW YORK AT OSWEGO | HUMAN – COMPUTER INTERACTION
- 7. Variety of Visualizations From this, it would be helpful to offer users the opportunity to visualize information in representations that can provide information at a glance as well as advanced visualizations that can provide additional insight into relationships amongst variables in the dataset. Classifiers Users had a hard time determining the difference between textures. No noisy fill patters or line styles. Others did not find the shapes helpful as identifiers because they were hard to differentiate between. Users indicated that they would like to be able view corresponding colors to the different star types as classified by shape. Others would like to select a group of data points and manipulate them separately. Limit the amount of options for users to encode texture and shape into classifiable variables. Color Users did not find the shapes helpful as identifiers because they were hard to differentiate between. Further, users also indicated that the colors of some data points were too similar and found it hard to differentiate between them at times. No saturated or bright colors State University of New York at Oswego – 7060 New York 104 – Oswego, New York 7 STATE UNIVERSITY OF NEW YORK AT OSWEGO | HUMAN – COMPUTER INTERACTION