Exploratory Visual Analysis in Large High-Resolution Displays

ExploratoryVisual Analysis in Large
High-Resolution Display Environments
Khairi Reda

• Society is generating data at ever
escalating rates
• The rise of the supercomputer
• Advances in data collection
instruments and techniques (e.g.,
high-throughput genome
sequencing)
• We can now afford to capture and
store larger quantities of information,
in the hope that we can learn
something unexpected from it
!2
background

background !3[Tukey and Wilk 66, Tukey 77, Tukey 80]
• Confirm / disconfirm a
particular hypothesis
!
• Typically involves statistical
tests and computation
modeling
!
• Perfect for computers
!
!
!
Confirmatory
analysis
• Look for unexpected patterns
and outliers
!
• Ask new questions!
!
• Not a fit task for computers
!
!
!
!
Exploratory
analysis
Hypothesis-driven
inquiry
Data-driven
inquiry

background !3[Tukey and Wilk 66, Tukey 77, Tukey 80]
• Confirm / disconfirm a
particular hypothesis
!
• Typically involves statistical
tests and computation
modeling
!
• Perfect for computers
!
!
!
Confirmatory
analysis
• Look for unexpected patterns
and outliers
!
• Ask new questions!
!
• Not a fit task for computers
!
!
!
!
Exploratory
analysis
Hypothesis-driven
inquiry
Data-driven
inquiry
New perceptual aids are needed to facilitate the
exploration of “big data” sources

• Cost of digital displays is decreasing rapidly
• Large high-resolution displays are
becoming more affordable
• Deployed in real-world scientiﬁc settings
• Collaboration between co-located teams
• Visual analysis and exploration of large-
scale datasets
Nanoscale materials science
Earth sciences
Ecology and
behavioral
biology
background [Reda et al. 13b, Febretti et al. 13, Reda et al 13c] !4
Could these display improve the rate
and quality of insight during exploratory
visual analysis? And how/why?
?

!5
information
space
visual exploration

!5
information
space
+ + =?
visual exploration

!6
visual exploration
temporal separation

!6
visual exploration
temporal separation spatial separation (i.e., juxtaposition)

!6
visual exploration
?
Aha!

!6
visual exploration
1. What is the effect of increasing the physical size and resolution of the
display on user behavior, during visual exploration?
2. And on insight acquisition?
3. Are there new design patterns for exploratory multi-view
visualizations on large high-resolution displays?
?
Aha!

!8
Low-level visualization tasks
(e.g., ﬁnd the state with the highest population growth between 1960 and 1990)
Task
completion

time
Data size
Strictlylinearscaling
User performance on a
big display
[Yost and North 06, Yost et al. 07, Ball et al. 07]related work

!9related work [Andrews et al. 10, Andrews and North 13]

theory !10
View1
display
View2
View3
View4

theory !10
View1
display
View2
View3
View4
time
sequential
access to
information

theory !10
View1
display
View2
View3
View4
View1 View2
View3 View4
display
time
sequential
access to
information

theory !10
View1
display
View2
View3
View4
View1 View2
View3 View4
display
time
sequential
access to
information
embodied,
non-sequential
access to information

theory !10
View1
display
View2
View3
View4
View1 View2
View3 View4
display
time
sequential
access to
information
embodied,
non-sequential
Cost (Temporal-separation) Cost (Spatial-separation)

theory !10
View1
display
View2
View3
View4
View1 View2
View3 View4
display
time
sequential
access to
information
embodied,
non-sequential
Cost (Temporal-separation) Cost (Spatial-separation)=
>
<

!11[Norman 02]
seven
stages of
action
Goal
System
Intention
Action
Execution Perception
Interpretation
Evaluation
Cost ~ Usability: the ease and learnability of a
human-made object
theory

!12[Lam 08]
Cost (temporal-separation)
theory
Form goal!
Visualization tool
Form physical sequence!
Execute physical sequence
Form system operators!
Perceive state
framework of
interaction
costs in
visualizations
Evaluate interpretation
Interpret perception

!12[Lam 08]
theory
framework of
interaction
costs in
visualizations

!12[Lam 08]
theory
framework of
interaction
costs in
visualizations
Interpret perceptionTemporal-view association

!12[Lam 08]
theory
framework of
interaction
costs in
visualizations
Interpret perceptionTemporal-view association
Mental map (re)-building
[Purchase et al. 07]
Cognitive integration
[Ratwani et al. 08, Plumlee & Ware 06 ]
}

!12[Lam 08]
theory “cognitive resistance”
Form goal!
Visualization tool
Execute physical sequence
Perceive state
framework of
interaction
costs in
visualizations

!13[Lam 08]
Perceive stateExecute physical sequence
Visualization tool
Form goal!
Cost (spatial-separation)
theory

!13[Lam 08]
Visualization tool
theory

!13[Lam 08]
Visualization tool
theory
Potentially strenuous
physical navigation
(head turns, walking, leaning)

!13[Lam 08]
Visualization tool
theory
Managing attention in a
more complex environment
Visually resolve a
more cluttered visualization
Potentially strenuous
physical navigation
(head turns, walking, leaning)

!14
• Study I: exploratory case study

• Comparative visual analysis of ensemble datasets on large
displays
• Real-world application + domain expert (behavioral ecology)
!
• Study II: comparative study

• Investigate the effects of increasing the size/resolution of the
display on user behavior and insight acquisition
• Simulated analysis task on a real-world dataset

!15
to nestto food
off-trail navigation?
Understanding the navigational strategies of Seed harvester ants
case study [Offord et al. 13]

!16case study
On-trail
ants West side ants East North south

!16case study
On-trail
ants West side ants East North south
design pattern:
query-by-example brush

!17
Findings: user strategy
case study
• Think-aloud protocol
• Verbal protocol analysis
• Coding scheme: Observations,
Hypotheses, and Decisions
• Interactions: Query by example,
Workspace management
Workspace
management
Query by
example
Hypothesis
formulation
Decision
making
Observing
outliers
layout-preserving
interactions
temporal separation of views

!18
Study II: Effects of increasing the display size and resolution on
user analytic behavior and insight acquisition
• Goals
• Measure effects of increasing the display size and resolution on quantity and quality of
discoveries made during visual exploration
• Understand variations in user behavior and analytic strategy induced as a result of using a
larger display with more pixels
• Study design
• Two display conditions (Small, Large)
• Between-subject design
• Scenario
• Open-ended visual exploration task: analysis of Chicago crime patterns between 2006 -
2012 (approximately 2.8 million crime incidents)
experiment

experiment !19
Visualization interface
[Cockburn et al. 08]
201220092006
TheftBurglaryNarcotics
overview map

experiment !19
design pattern:
“seed and grow”
201220092006
overview map

!20
participant
experimenter
4
meters
wide-angle
video camera
keyboard & mouse
paper
notepad
CAVE2
experiment [Reda et al. 13b]

!21
Experimental conditions
small
!
3 x 4 panels
12 Megapixels
40 degree FOV
large
!
13 x 4 panels
54 Megapixels
190 degree FOV
experiment

!22
experiment
• Participants
• 10 unpaid volunteers (4 female) recruited from EVL
• Distributed evenly under the two experimental conditions
• Procedure
• 15-minutes training
• 2.5 hours of open-ended exploration
• Semi-structured debrieﬁng interview
• Verbal protocol analysis
• Coding scheme: Observation, hypothesis, question, goal, comment.
• 5-points quality score: lower-score refer to isolated insights. Higher scores
imply broader, more integrative insights

!23
Results: exploration time
experiment

!23
experiment
0!
20!
40!
60!
80!
100!
120!
140!
large! small!
minutes!
Average length of exploratory
activity!

!23
experiment
0!
20!
40!
60!
80!
100!
120!
140!
large! small!
minutes!
Average length of exploratory
activity!
t(8) = 33.5, p < .01

!24
Results: observations
experiment

!24
experiment
0!
40!
80!
120!
160!
200!
O1! O2! O3! O4! O5! All!
Average number of observations!
large!
small!
* * * *

!24
experiment
0!
40!
80!
120!
160!
200!
O1! O2! O3! O4! O5! All!
large!
small!
* * * *
𝝌2(4, N=1327) = 263.3, p < .001

!24
experiment
0!
40!
80!
120!
160!
200!
O1! O2! O3! O4! O5! All!
large!
small!
* * * *
0!
0.2!
0.4!
0.6!
0.8!
1!
1.2!
1.4!
1.6!
O1! O2! O3! O4! O5! All!
observation/minuteofanalysis!
Observation rate!
large!
small!
* * *
𝝌2(4, N=1327) = 263.3, p < .001

!25
Results: hypotheses
experiment

!25
Results: hypotheses
experiment
0!
5!
10!
15!
20!
25!
H1! H2! H3! H4! H5! All!
Average number of hypotheses!
large!
small!

!25
Results: hypotheses
experiment
0!
5!
10!
15!
20!
25!
H1! H2! H3! H4! H5! All!
large!
small!
𝝌2(4, N=145) = 67.3, p < .001

!25
Results: hypotheses
experiment
0!
5!
10!
15!
20!
25!
H1! H2! H3! H4! H5! All!
large!
small!
0!
0.05!
0.1!
0.15!
0.2!
0.25!
H1! H2! H3! H4! H5! All!
hypothesis/minuteofanalysis!
Hypothesis formulation rate!
large!
small!
𝝌2(4, N=145) = 67.3, p < .001

!26
Results: quantity and rate of insights
experiment
• Comparable rate of insight between the two display conditions
• Participants chose to spend 35 minutes extra time on average exploring the
dataset with the large display
• This may have caused participants to make more observations with the
large display

!26
Results: quantity and rate of insights
experiment
• Comparable rate of insight between the two display conditions
• Participants chose to spend 35 minutes extra time on average exploring the
dataset with the large display
• This may have caused participants to make more observations with the
large display
minutes into activity
commutativeinsights

!27
experiment
Results: quality of insights

!27
experiment
insight
“breadth”
probability
• The large display is more likely to elicit broader, more integrative
insights.

!28
Results: user behavior
experiment

!28
experiment
participant S1
(small)

!28
experiment
participant S1
(small)
participant L5
(large)

!29
experiment
p(large) - p(small)

!29
experiment
p(large) - p(small)
• Signiﬁcant tendency to “integrate information / continue to make observations” with
the large display
• More frequent transitions to “form goal” (36% increase, non-signiﬁcant)
• Suggests the formulation and pursuit of more exploratory goals with the large
display

!30
Discussion: human limits
experiment
• Some complaints suggest information overload situations with the large display
• “It’s hard to look over so much! It’s so hard to compare so many things.”

• Difﬁculty in integrating information across spatially disparate views
• “By the time I ﬁnished turning my head to the other side I would forget [the contents of
the previous view]”

• Participants on the large display seem to limit themselves to 6-7 columns (87-102
degrees of visual angle)

!31
Discussion: summary
RQ1 - What is the effect of increasing the display size/resolution on user behavior?
• Significant increase in the length of the exploratory activity.
• Suggests a tendency to pursue more ambitious exploratory goals with the large display
• Small display participants seem to resist exploration when they did not have a priori intuition: “I only bothered to look
at the years when I knew something about an area– like Cabrini Green and the Taylor area”
!
• Significant tendency to integrate observations / continue to derive new observations!
RQ2 - What is the effect of increasing the display size/resolution on insight acquisition?
• Significant increase in the number of observations reported
• Slight increase in the number of hypotheses formulated (not significant)
• Comparable rates of insight (for observations and hypotheses)
• Significant tendency to generate higher-level, more integrative insights.
!
RQ3 - Are there new design patterns for multi-view based visualization on large displays?
• “Seed and grow” design patter
• Loose view coordination model to support multiple exploratory threads
• Lens metaphor to facilitate the exploration of disparate parts of the information space

!32
Scientiﬁc reasoning through dual search
outlook [Klahr and Dunbar 88]

!33
Scientiﬁc reasoning through dual search: experimenters
outlook
Hypothesis
space
experiment /
Information
space
[Klahr and Dunbar 88]

observations
!33
outlook
Hypothesis
space
experiment /
Information
space

observations
!33
outlook
Hypothesis
space
experiment /
Information
space
generalize

!34
Scientiﬁc reasoning through dual search: theorists
outlook
Hypothesis
space
experiment /
Information
space

!34
outlook
Hypothesis
space
predict
experiment /
Information
space

!34
outlook
Hypothesis
space
predict
collect
evidence
experiment /
Information
space

evidence
!34
outlook
Hypothesis
space
predict
collect
evidence
experiment /
Information
space

evidence
!34
outlook
Hypothesis
space
predict
collect
evidence
evaluate
evidence
experiment /
Information
space

evidence
!34
outlook
Hypothesis
space
predict
collect
evidence
evaluate
evidence
re-frame
experiment /
Information
space

!35contributions
theory
[c2]
design
patterns
[c3]

!35contributions
apply
theory
[c2]
design
patterns
[c3]

!35contributions
apply
theory
[c2]
design
patterns
[c3]
case study +
experiment
[c4-5]

!35contributions
apply instantiate
theory
[c2]
design
patterns
[c3]
case study +
experiment
[c4-5]

!35contributions
apply instantiate
validate
theory
[c2]
design
patterns
[c3]
case study +
experiment
[c4-5]

!35contributions
apply instantiate
validate
theory
[c2]
design
patterns
[c3]
case study +
experiment
[c4-5]
scientiﬁc
reasoning
hypothesis
observations
[c6]

!35contributions
apply instantiate
validate
reﬂect
theory
[c2]
design
patterns
[c3]
case study +
experiment
[c4-5]
scientiﬁc
reasoning
hypothesis
observations
[c6]

!36
future work
• New design patterns?
• Desktop has “overview, zoom, detail-on-demand” [Shneiderman, 96]
• “Spread, eye-ball, coalesce”?
• Design space for multi-view-based visualizations on large displays
• Collaborative visual analytics
• This is where the cost investment makes more sense.
• Applications and real-world case studies

Exploratory Visual Analysis in Large High-Resolution Displays

background
Why visualize data?
!40
visualization statistics
I II III IV
x mean 9 9 9 9
y mean 7.50 7.50 7.50 7.50
x variance 10 10 10 10
y variance 3.75 3.75 3.75 3.75
x/y correlation 0.81 0.81 0.81 0.81
[Anscombe’s quartet]

background
Why visualize data?
!40
visualization statistics
I II III IV
x mean 9 9 9 9
y mean 7.50 7.50 7.50 7.50
x variance 10 10 10 10
y variance 3.75 3.75 3.75 3.75
x/y correlation 0.81 0.81 0.81 0.81
[Anscombe’s quartet]
Contained within the data of any investigation is information that can yield conclusions to
questions not even originally asked. That is, there can be surprises in the data… To regularly
miss surprises by failing to probe thoroughly with visualization tools is terribly inefﬁcient
because the cost of intensive data analysis is typically very small compared with the cost of
data collection.
-William Cleveland, the Elements of Graphing Data

!41
intuition,
prior hypotheses,
experience
!
!
Visualization
spontaneous
observations,
new hypotheses
observations: unit of knowledge acquired from the visualization
!
hypothesis: conjecture that cannot be directly inferred from the
visualization
visual exploration
{insight
form exploratory
goals
interpret
visual patterns
[Liu and Stasko 10, Treisman 86]

!42
Alternate projections
of the same information
Multiple visualization
states (i.e., exploratory
threads)
Different subsets of
information
(aka, small-multiples)
visual exploration
http://ﬂowingdata.com/tag/small-multiples/
[Wang Baldonado et al. 2000]
Vis!
workspace 1
Vis!
workspace 2
Vis!
workspace 3

!43
Visualization
• User performance kept pace with increasing data and screen size [Yost
et al. 07]
• Increased physical navigation over virtual navigation [Ball and North 05,
Ball et al. 07]
• Mixed results in map-related tasks; some interfaces are poorly-suited to
large displays (e.g. context + focus lenses) [Jakobsen and Hornbæk 11]
Intelligence
analysis
(text)
• Memory externalization, semantic spatial arrangement (users attach
meaning to space), Schematization [Andrews et al. 10]
• Automatic inferring of semantics from interactions [Endert et al. 12b]
Ofﬁce
environment
• Improved spatial cognition + embodied memory [Tan 04]
• Reduced gender performance disparities in virtual navigation [Tan 03b]
• Reduced window switching, and reduced user frustration [Ball and
North 05]
• Increased productivity in cognitively demanding ofﬁce work [Czerwinski
et al. 03, Bi et al. 09]
related work

!44
High-res / reduced FOV
[Ball and North 08]related work
Low-res / wide FOV
• High-resolution + physical navigation is more important to
improving user performance

!44
High-res / reduced FOV
[Ball and North 08]related work
Low-res / wide FOV
• High-resolution + physical navigation is more important to
improving user performance
context + focus displays
[Baudisch et al. 02]

!45
action
perception
visual exploration

!46
action
perception
data mining,
ﬁltering,
transformation,
re-projection
induction,
deduction,
analogical
thinking
T1
T2
information
space
insights
visual exploration [Sedig et al. 12]

!46
action
perception
data mining,
ﬁltering,
transformation,
re-projection
induction,
deduction,
analogical
thinking
T1
T2
information
space
insights
Temporal
separation
visual exploration [Sedig et al. 12]

!47
theory
!
!
!
!
!
!
!
!
!
!
View1 V2 V3 V5V4
[Maxcey-Richard and Hollingworth 13]
unexpected discoveries
Cost (temporal-separation): bottom-up costs
time

!47
theory
!
!
!
!
!
!
!
!
!
!
View1 V2 V3 V5V4
[Maxcey-Richard and Hollingworth 13]
unexpected discoveries
Cost (temporal-separation): bottom-up costs
time
P(View1)

!48
theory
• Top-down costs

• Discourage the formation of new exploratory goals
• `Tunnel vision’ phenomenon where exploration is focused on isolated subsets
in the information space
!
• Bottom-up cost

• Overuse of visual working memory to retain visual patterns of interest
• Reduction in the probability of making spontaneous inferences relating to
patterns deposited in temporally-separated views
Cost (temporal-separation): summary

!48
theory
• Top-down costs

• Discourage the formation of new exploratory goals
• `Tunnel vision’ phenomenon where exploration is focused on isolated subsets
in the information space
!
• Bottom-up cost

• Overuse of visual working memory to retain visual patterns of interest
• Reduction in the probability of making spontaneous inferences relating to
patterns deposited in temporally-separated views
Cost (temporal-separation): summary
?

!49
Screen surface
Time
2D movement
Trajectory
case study

experiment !50
201220092006
overview map

experiment !50
design pattern:
“seed and grow”
201220092006
overview map

experiment !50
Density
of crime incidence
Yearly, weekly, daily
crime trends
design pattern:
“seed and grow”
201220092006
overview map

!51
experiment
structure layoutfreeform layout
large
small

!52
experiment
• 3 “mental states”
• Make observation
• Form goal
• Formulate hypothesis
• 2 types of interaction
• Layout-preserving (brush-and-link / pan map)
• Layout-changing (create new view, changing view contents)

!53
Results: user behavior / strategy
experiment
average
(small)
average
(large)

!54
0!
1!
2!
3!
4!
5!
large!
small!
usability utility
usability / utility
experiment

!55
experiment
Do higher-level insights occur at a later time in the activity?

!55
experiment
minutes into activity
levelofinsight
Do higher-level insights occur at a later time in the activity?

!56
Form goal!
Visualization tool
Execute physical sequence Perceive state

!57contributions
1. Theory of how interaction costs affect user behavior in exploratory visual analysis
• `Tunnel vision’ phenomenon in situations involving large amounts of data
• Costs are elevated due to excessive temporal-separation of views
2. Effects of increasing the size and resolution of the visualization interface on user
behavior and insights acquisition
• Increased user investment / exploration
• More observations
• Formation of higher-level, more integrative insights
3. Design patterns for large high-resolution displays
• Query-by-example brush for the analysis of ensemble data
• “Seed and grow” pattern: loose view coordination model with lens metaphor

Exploratory Visual Analysis in Large High-Resolution Displays

More Related Content

Similar to Exploratory Visual Analysis in Large High-Resolution Displays (20)

Recently uploaded (20)

Exploratory Visual Analysis in Large High-Resolution Displays