Reprioritizing the Relationship Between HCI Research and Practice: Bubble-Up and Trickle-Down Effects
2 likes543 views
This document discusses the fractured relationship between HCI research and practice, proposing a model that describes the 'bubble-up' of practitioner insights informing research and the 'trickle-down' of academic theory into practice. Through interviews with interaction designers, it explores practitioners' use of design methods and highlights how current theories often fail to address real-world design problems. The authors emphasize the need for a better understanding of practice to improve the connection between research and its application in the field.
![Reprioritizing the Relationship Between HCI Research and
Practice: Bubble-Up and Trickle-Down Effects
Colin M. Gray
Indiana University Bloomington
201 N. Rose Avenue, #2276
Bloomington, IN 47405 USA
comgray@indiana.edu
Erik Stolterman
Indiana University Bloomington
901 E. 10th St.
Bloomington, IN 47405 USA
estolter@indiana.edu
Martin A. Siegel
Indiana University Bloomington
901 E. 10th St.
Bloomington, IN 47405 USA
msiegel@indiana.edu
ABSTRACT
There has been an ongoing conversation about the role and
relationship of theory and practice in the HCI community.
This paper explores this relationship privileging a practice
perspective through a tentative model, which describes a
“bubble-up” of ideas from practice to inform research and
theory development, and an accompanying “trickle-down”
of theory into practice. Interviews were conducted with
interaction designers, which included a description of their
use of design methods in practice, and their knowledge and
use of two common design methods—affinity diagramming
and the concept of affordance. Based on these interviews,
potential relationships between theory and practice are ex-
plored through this model. Disseminating agents already
common in HCI practice are addressed as possible mecha-
nisms for the research community to understand practice
more completely. Opportunities for future research, based
on the use of the tentative model in a generative way, are
considered.
Author Keywords
Interaction design; design practice; design methods; com-
munity of practice.
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g., HCI):
Miscellaneous.
INTRODUCTION
In the field of HCI there has been an ongoing discussion
about the relationship between academic and professional
practice. The lack of recognition and adoption of research
results among practitioners has been seen by some as a
problem for academic research [10, 23, 26], while at the
same time, practitioners have expressed that research is not
addressing their everyday problems and therefore not offer-
ing suitable tools for their needs [23, 27]. This study focus-
es on how this current state of affairs impacts the practice
community, and how researchers have the potential to in-
form practice by first understanding it more fully.
In this paper we will explore two aspects of the relationship
between academic research and practice. We have labeled
the two as the bubble-up effect and the trickle-down effect.
The bubble-up effect describes the efforts of the practice
community—and ideally the academic community as
well—to refine and abstract situated knowledge and prac-
tice of methods, tools, or concepts into refined theory and
defined tools and methods. The trickle-down effect follows
the more traditional research tradition, denoting and de-
scribing the way adaptation of research and theory is com-
monly seen to take place in design practice, including the
opportunistic use of methods, tools, or concepts that origi-
nate in an academic community.
Even though the bubble-up and trickle-down phenomena
indicate an exchange of expertise and knowledge between
research and practice, it is often not a well-functioning rela-
tionship where both sides strongly benefit from the other
side, or even share core values about what constitutes prac-
tical knowledge. This mismatch in the relationship between
research and practice is problematic for several reasons: it
means that researchers in many cases spend time and ener-
gy devoted to research aimed at supporting practice that is
largely ignored by practitioners; at the same time, profes-
sional practice is often shaped by institutional and tradition-
al norms and values that have the potential for improvement
through research and practitioner partnerships. This mis-
match can be understood as a general devaluing of the “oth-
er side,” that is, researchers view practice as uninformed
while practitioners view research as not being in touch with
the reality of real world design.
[27] argues that there is a need in HCI research to focus
more on existing practice, understood on its own terms. In
that study, the relationship of the designer to their tools in
the act of designing was found to be significantly more
complex than previously assumed. This intentional focus on
practice revealed that conceptions of practice in academ-
ia—which are then codified in theory and pedagogy—are
frequently not grounded in any study of how designers
practice “in the wild.” The practice community that re-
searchers project does not necessarily represent the way
HCI practice is actually conducted (Figure 1). This is not a
new phenomenon; Rogers [23] reports on several earlier
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full cita-
tion on the first page. Copyrights for components of this work owned by others than
the author(s) must be honored. Abstracting with credit is permitted. To copy other-
wise, or republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee. Request permissions from permissions@acm.org.
DIS 2014, June 21–25, 2014, Vancouver, BC, Canada.
Copyright is held by the owner/author(s). Publication rights licensed to ACM.
ACM 978-1-4503-2902-6/14/06..$15.00.
http://dx.doi.org/10.1145/2598510.2598595
studies that show the same pattern where research results
are not adopted by practice in the way intended by re-
searchers. We see the mismatch, shown in Figure 1, as a
serious problem for the HCI research community.
Figure 1. Practice in situ and projected practice.
There is even less research to be found in HCI addressing
ways practice might positively influence research, that is,
what we label the bubble-up effect, although practitioners
have developed a wide range of professional conferences,
blogs, and other tools to meet these needs internally (Figure
2).
Figure 2. HCI research as processes of
trickle-down and bubble-up
This paper reports on our attempts to develop a better un-
derstanding of the relationship between research and prac-
tice, in particular from the perspective of practice and from
practitioners. To explore some aspects of these phenomena
we developed an interview study that documented general
beliefs about and use of methods by design practitioners,
and their knowledge and use of two specific methods—
affordances and affinity diagramming—in their design
practice. This research occurred in parallel with additional
exploration of how researchers as developers of design
methods conceive of their use in practice.
We first discuss the existing relationship between academic
research and practice as seen through the lens of the re-
search tradition. We then discuss the history and context of
two specific design methods, using this discussion as a
framing to explore traditional conceptions of method use in
practice and research traditions. Finally, we provide find-
ings and discussion from interview participants in relation
to these specific design methods, with implications for the
relationship between academic research and practice.
THE DYNAMIC RELATIONSHIP BETWEEN THEORY
AND PRACTICE
The connection between theory and practice is seen by
many as vital, but often tenuous due to the intersection of
competing communities of practice [6, 11, 16]. There are
different ways to understand the relationship between these
communities depending on focus and purpose. We have, in
this project, worked with a tentative understanding of this
relationship as the flow of information from one community
of practice to another—along dimensions of appropriation
[23] and abstraction of in situ detail.
Rogers [23] traced the dimension of appropriation as it re-
lates to the relationship between theory and practice, but
only in the application of existing research to practice, that
is, the appropriation of theory through use and adaptation in
design practice. We will focus more strongly on the impli-
cations of stimulating a flow from both sides, with equal
representation and valuing of communities.
The relationship between research and practice can refer to
many different potential areas of concern, such as practical-
ities (e.g., time, resources), competence, skills, and organi-
zation [6]. In our research we focused on the way design
methods are understood, developed, and used in the two
communities of practice. We define design methods broad-
ly, as any intellectual or practical support that a practitioner
might use to support the design process in a positive way,
encompassing everything from everyday methods, tech-
niques, and tools for idea generation and collaboration, such
as the "pen and paper," whiteboard, brainstorming, dialogu-
ing, to methods and applications for sketching and refining
ideas. For convenience, we label the flow of information
about design methods from practice to research as bubble-
up and the flow of information from research to practice as
trickle-down (Figure 2).
Based on this tentative model of the relationship between
research and practice and a preliminary understanding of
the two notions of bubble-up and trickle-down we devel-
oped our research design. We first introduce our approach
and the two design methods we decided to focus on. We
then present our interview study and our findings. To con-
clude, we return to our tentative model and our two con-
cepts and how they can be further developed in a generative
way to inform future research.
OUR APPROACH
As a way to examine how practitioners view design meth-
ods, as defined in this paper, we decided to focus on two
fairly well known design methods: affinity diagramming
and affordances. We are well aware of the fact that these
two methods are not necessarily methods in any strict
sense; although affinity diagrams have been developed into
more formal variations over time [2, 12] and understand-
ings of affordances have evolved over time [13].
Our choice of these two methods were based on their dif-
ference in apparent source, with affordances coming from
ecological research methodology and affinity diagrams
most often seen as coming from a business and marketing
context (although the true source is cultural anthropology
[1]). These methods also represent a range of method types:
from generative and interpretative (affinity diagramming) to
descriptive and analytic (affordances).
PRACTICE
IN SITU
PROJECTED
PRACTICE
COMMUNITY
≠
PROJECTED
PRACTICE
COMMUNITY
RESEARCH RESEARCH
A. Trickle-Down
Researchers create methods for use in practice
B. Bubble-Up
Researchers study of how practitioners use methods
within the structures of the research community
(if at all).
A B
We researched the history and background of each method,
including: attention to the originating field or discipline,
contributing authors or scholars, the first entrance of this
method into HCI, and the current state of the method in
HCI literature. We have no ambition to be comprehensive
or to give a complete description of each method, since that
is not the purpose of our work and we also expect most
readers to be familiar with them. Instead, we provide some
background information as a point of comparison from
which interview data can be contrasted and viewed.
AFFINITY DIAGRAMMING
Affinity diagramming is commonly used as a tool in busi-
ness to organize and cluster ideas or concepts. It appears to
have been introduced to the Western world during the
movement to Total Quality Control (TQC) in Japan, and
was published as one of seven key business tools [28]. Alt-
hough this tool became popular in a business context, it was
originally known (and is still known in some Asian con-
texts) as the KJ Method, which originates as a method for
sorting ethnographic data.
The KJ Method was created by Japanese cultural anthro-
pologist Jiro Kawakita, and was originally created as a way
to analyze data obtained through ethnographic methods in
the 1960s [14]. Early examples of this method appear to be
solitary without significant iteration, used as a way to “let
the facts speak for themselves,” grouping facts based on
relationships that were perceived to join them [1]. After
individual relationships were established, groupings were
narrowed down to a reasonable number, generally ten or
less. While the method is still used this way in some con-
texts, it is also used as a generative method, instigating the
generation of data from stakeholders, and then evaluating
relationships and connections between concepts, instead of
only a way to evaluate a known data set.
Beyer & Holtzblatt (2) are some of the first to discuss this
method within the HCI community, introducing major con-
cepts and procedures for carrying out this method from the
original TQC books [28; see also 18, 30]. Very little re-
search has been carried out specifically on the use or adap-
tation of this method within the HCI context, aside from the
creation of digital tools to support the core activities of the
method [7]. However, various forms of Post-It note based
diagrams seem to be common in both educational and pro-
fessional settings. To what extent this use is true to the
method of affinity diagrams is less known.
AFFORDANCES
Rogers [23] provides an excellent historical overview of the
notion of affordance, from its early roots in ecological
methodologies within the psychology community [8, 9, 25]
to the importation of the key idea of affordances into the
HCI community through the work of Don Norman [20].
The notion of affordances is defined by Norman as “…the
perceived or actual properties of the thing, primarily those
fundamental properties that determine just how the thing
could possibly be used. […] Affordances provide strong
clues to the operation of things. […] When affordances are
taken advantage of, the user knows what to do just by look-
ing: no picture, label, or instruction is required.” [20]
However, today, there are many different definitions and
the way the concept is used differs widely [13]. The under-
lying concept of affordances is used both with and without
its historical roots.
In recent HCI scholarship, the notion of affordances has
been re-examined. According to Kaptelinin & Nardi [13],
the early work of Gibson informs the concept of affordanc-
es, describing the notion of interaction between animals and
the environment as a primary source for insights. They dis-
cuss the potential for mediated action as a way to recast
affordances as a method for investigating “distinctly human
uses of interactive technologies.” [13]. This re-examination
of the concept attracted substantial interest at the session
where it was presented at CHI in 2012.
According to Rogers [23] the notion of affordances is an
example of a theoretical construct that has transitioned from
research to practice while, in the process, losing its com-
plexity and richness. Rogers notes that many practitioners
recognize the concept and even “use” it without knowing its
background or history.
INTERVIEW STUDY
The purpose of our interview study was to find out how the
two design methods we selected—affinity diagrams and
affordances—are currently understood and used by practi-
tioners. We were particularly interested to discover how
much practitioners know about the background and history
of these methods, and where and how they learned about it,
and to what extent they use it—that is, the trickle-down
effect. It is important to note that this interview approach
inherently defines and values these methods from a research
perspective by using labels from that tradition; issues of
labeling become clear in the analysis of the interview data.
Implementation and use of a method in practice, regardless
of knowledge of the history or background of a method
would be sufficient to see it as an example of the trickle-
down effect, rendering these aspects of the method suffi-
cient, but not necessary. Similarly, the “core” of these
methods may have also been developed and used by practi-
tioners without knowledge of the “proper” title. This simi-
larity between practice-originated and research-originated
methods would not be discovered unless practice-centered
research was carried out. We also wanted to find out if the
practitioners were aware of any professional practices that
have been developed and manifested in more formalized or
structured ways or informed attempts to construct design
methods based on practice and shared with a professional or
research community—that is, the bubble-up effect.
Data Collection
We selected 13 practitioners from 12 companies, beginning
with an opportunistic sample of HCI alumni from a large
university, and then snowballing to include colleagues and
other figures in their professional networks. All of the prac-
titioners we located and subsequently interviewed worked
as an interaction designer, user experience designer, or user
researcher in some way. They had a range of professional
experience: several practitioners had 10 or more years of
experience, while the majority had worked 3-10 years.
The practitioners had a variety of educational backgrounds:
of the 13 interviewees, eight had an HCI-oriented educa-
tion, three had a technical or engineering background, and
one had a visual design background. Five of the interview-
ees were female, while the remaining eight were male.
A phone or face-to-face interview was scheduled with each
participant. Each interview followed a semi-structured for-
mat, and was audio recorded to facilitate further analysis. In
addition, a dedicated researcher took detailed notes either
during or shortly after the completion of the interview.
A subset of the interview participants (Table 1) was select-
ed based on numerous criteria to preserve the diversity of
the original sample, including years of experience, voca-
tional background, nationality, and gender. This subset is
used to facilitate a more detailed discussion of our findings.
Name
(Pseudonym)
Years of
Experience
Vocational
Background
Cheryl 10+ Business
Abbie 20+ Computer Science
Vicki 4 HCI
Phil 2 HCI
Jagdish 3 HCI
Table 1: Subset of interview participants.
Findings
Our interviews covered a broad set of questions aimed at
understanding professional practice in the field. In this par-
ticular analysis we have focused on a small portion of the
overall interview. These questions do not probe all aspects
of the relationship between theory and practice, but they do
provide us with some insights into the way practitioners
think and reflect when it comes to design methods.
Where did they learn methods?
Some participants learned methods in graduate school, but
in most cases dissemination of methods was through en-
counters with colleagues in a design context. Phil specifi-
cally noted his formal education in HCI as informing his
knowledge and use of methods, while other designers who
had practiced in the field without graduate work in HCI
relied more on their colleagues, professional conferences,
or publications for their understanding of design methods.
Due to the relative newness of the field of interaction de-
sign and the presence of many in the field who are trained
primarily in other fields, a strong cross-pollination between
related disciplines such as marketing, visual design, and
business lead to the adaption and adoption of new methods.
Cheryl experienced this phenomenon, tracing an important
method for doing “guerilla research” that had been in-
formed by her boss, who had learned this method of re-
search from a marketing company. Cheryl has since written
about related methods, further disseminating these ideas
into her company and practice community.
Participants sometimes felt that they had made a method up
or felt that it was common sense approach, and then found a
name to apply to that idea or concept later on. Abbie dis-
cussed the “common-sense” nature of affinity diagramming
as a basic way of filtering data. She had used this general
method prior to learning about it in a more formal sense at
an early CHI conference.
What is their conception of research and theory in their prac-
tice of design?
While most of the practitioners we interviewed seemed
aware of the research and academic community, they did
not attempt to forge a connection between their community
of practice and the academic community or professional
academic community (e.g., ACM/CHI). Cheryl discussed
her impression that CHI was generally populated by gradu-
ate students, and that she believed in “spreading the wealth”
by sending her employees to a broad range of more practi-
tioner-focused conferences, looking to her employees to
discover which events were most valuable. This shift to
practitioner-focused events was a general trend from our
interviewees, with a more direct emphasis on justifying the
expenditure to attend these conferences through real and
tangible gains in applicable methods.
Abbie reflected on the role of professional conferences,
especially CHI, noting that there was too much “big lan-
guage” and not enough “beer and steak” interaction with
peers. This shift to the theoretical and away from the practi-
cal was a main reason she has shifted to other venues over
time, as she describes herself as an “applied person.” Most
interviewees seemed aware of theory, and how it might
inform patterns of research or testing, but as practitioners,
they mentioned the constraints of client budgets, and how
the budgeting process often came in direct conflict with the
“proper” application of theory or research-driven approach-
es to design or user research.
Cheryl discussed the frequent conflict between the need for
research, and the lack of willingness from clients to pay for
this research. As a result, her company has drifted from
more traditional research methods to high-value, low-cost
methods of user research, like guerilla research.
How do they utilize methods in their design process?
According to almost all of our participants, designers want
methods that are easy to explain to their clients and easy to
visualize, for the purpose of assisting them in communi-
cating their design ideas. Ease of explanation and ease of
visualization often seem to go hand-in-hand, suggesting that
methods can be a powerful tool not only during the design
process, but also to facilitate communication about design.
Jagdish noted the use participatory design methods to
quickly engage clients and designers around core issues,
including them in key design decisions within the interface.
In conjunction with engagement or communication, level of
detail was a concern that Phil surfaced, as he related that the
“state of finality” influenced the designers and clients he
worked with, and thus preferred mediums and tools that
didn’t focus on details (e.g. Photoshop, Illustrator).
The practitioners seem to be highly opportunistic in their
selection and use of methods, often selecting methods based
on context, time and tools available, and designer
knowledge. Phil used methods in a highly opportunistic
way, selecting methods primarily based on the proximity of
the individual: using a document or Microsoft PowerPoint
for people who were unavailable, while preferring to use
wireframes with markers for someone in the same office. In
contrast, Abbie and Jagdish seemed to focus more tightly
on one set of methods—mental models for Abbie and usa-
bility testing and iteration for Jagdish—that they applied
opportunistically, varying by fidelity and approach given
the specific design context.
Designers also adapt methods as needed for a given design
situation, often seemingly unconsciously, to meet the con-
straints or requirements of the specific design problem be-
ing addressed. Vicki discussed her use of “scrappy” design
methods, noting that many “textbook” methods required
more time than she had to do “the right way,” so adapting
methods was often the only way to carry out a given meth-
od in practice. She further described the importance of im-
plicitly understanding your audience, and using appropriate
terms and methods to communicate with that audience. In
particular, she described the differences between sharing
information with other designers versus working with peo-
ple from marketing or their human factors lab, highlighting
the importance of speaking a consistent language with the
people you are communicating with, “translating” termi-
nology where appropriate between disciplines.
What do they know about the original intent or historical
context of the method?
For other participants, conceptual knowledge of the ideas
behind a method was most important, not knowledge of
specific “traditional” methods or processes known by a
certain name. The company Cheryl works for adapted affin-
ity diagramming based on experiences of some of their de-
signers, using more defined division of labor between de-
signers (including moderation) and “throw[ing] the post-it
note over their shoulder” after each idea is complete. This
type of adaptation, either corporately or in a specific design
situation, was quite common across all of our participants.
Most importantly, the interviewees seemed to be focused on
when to use design methods rather than basing their use on
the origin or historically informed use of a method. Phil
related an instance of a semantic differential, which was
originally referred to as an affinity diagram in our inter-
view, but was then retracted. Later in the conversation, Phil
demonstrated a robust hybrid method that he and a col-
league had developed, which included many ideas from
affinity diagramming. It is clear that knowledge of the
origin of a given method, or even prescribed steps to ac-
complish that method, is not a barrier to using the concep-
tual idea behind a method in a specific design context.
Yes No N/A
Affinity Diagramming
Concept 10 2 1
5%
Historical Context 0 12 1
6%Affordances
Concept 10 1 2
3%
Historical Context 3 8 2
3%
Table 2: Participant knowledge of methods
In Table 2, the participants’ knowledge of two selected
methods is detailed. N/A refers to participants who were not
explicitly asked, or did not explicitly answer, the question.
While most participants were comfortable interacting with
the general concept behind each method, confirmed by a
discussion of core ideas from the methods in their design
practice, their knowledge of the historical background or
context of the method was more limited. Even in the case of
affordances, where the participants knew some historical
context surrounding affordances, their knowledge was lim-
and detailed implementation of the original method, these
designers also seemed to be more willing to adapt and ap-
propriate these methods in practice.
DISCUSSION
In our discussion we will focus on the relation between
research and practice that emerged from our interviews,
primarily the notions of appropriation, abduction, and dis-
seminating agents. We will then return to our tentative
model and reflect on how our findings can be interpreted in
the light of the model and the concepts of bubble-up and
trickle-down. We will also add the notion of cycle-around
and discuss the concept of design judgment.
Through our analysis of the interviewee responses in rela-
tion to our tentative model presented here (Figure 3), ap-
propriation and abduction were identified as core activities
in both directions. Concepts discussed in the text are linked
to the model where relevant.
We found that appropriation of research and theory (A) is
frequent, especially in the application and context-of-use
for specific methods by practitioners, resulting in abduction
of research methods vis-à-vis real world contexts, while not
necessarily in the form expected by those who develop new
methods and tools. The appropriation is in many cases radi-
cal. Sometimes only core aspects of a method are preserved
and translated into actual practice.
Appropriation is found in the intentional or unintentional
reshaping of existing methods by individual practitioners.
We found this appropriation to be an essential part of using
methods, not necessarily a misuse of the original method.
This alters the original meaning of what it means that a
method is “used” in practice. When methods are radically
appropriated in an abductive sense, even though they are
originally highly formalized, they are still “used” and some-
times emerge as an even richer method in situ.
We also found that the lack of communication between
researchers and practitioners, in the eyes of the interview-
ees, leads to inaccurate and incomplete abstractions of
practice (Figure 1). This perceived lack of understanding of
design practice among researchers seems to be the core
reason why the practitioners do not show any serious inter-
est in investigating new methods developed in academia.
Disseminating Agents
Even though appropriation of academic results does not
seem to be common among practitioners, it does happen.
Based on the interviews we conducted, a number of dissem-
inating agents were identified that commonly bring design
methods into practice. While research and theory are often
assumed to be connected—at least peripherally—to prac-
tice, none of the practitioners we interviewed relied on any
traditional academic sources for information regarding new
methods or perspectives on tool use.
Disseminating agents seemed instead to be linked to indus-
try norms of employment, especially regarding which pools
of people work in a given segment of the industry, with
practitioners informing their own community in an educa-
tive sense (as shown with E in Figure 3). For example, in
more traditional graphic design or marketing fields (even if
they are UX centered), prevailing norms of those industries
tend to dominate. In newer, UX-focused teams, another
balance of people and sources of information dominates.
Regardless of industry segment, primary disseminating
agents were people (e.g., coworkers and colleagues from
professional groups and associations) and Internet discov-
ery through blogs, news articles, Twitter, and trade publica-
tions available online. Of particular interest is the role of
coworkers in shaping or redefining a person’s feelings and
knowledge about methods. Some of our interviewees
learned methods from specific coworkers, or in the context
of on-the-job training, while others learned methods more](https://image.slidesharecdn.com/dis14graystoltermansiegelpechakucha-140626160645-phpapp01/85/Reprioritizing-the-Relationship-Between-HCI-Research-and-Practice-Bubble-Up-and-Trickle-Down-Effects-6-320.jpg)
Reprioritizing the Relationship Between HCI Research and Practice: Bubble-Up and Trickle-Down Effects
- 1. REPRIORITIZING THE RELATIONSHIP BETWEEN HCI RESEARCH & PRACTICE Colin M. Gray Iowa State University! " Erik Stolterman Martin A. Siegel Indiana University Image courtesy of Venu Gopal https://www.flickr.com/photos/visualsymphony/6233131506
- 2. THE RELATIONSHIP BETWEEN RESEARCH & PRACTICE IS BROKEN Image courtesy of Jon Wiley https://www.flickr.com/photos/jonwiley/3169246516
- 3. Image courtesy of Jon Wiley https://www.flickr.com/photos/jonwiley/3169246516 PRACTICE ACADEMIA
- 4. Image courtesy of Austin Kleon https://www.flickr.com/photos/evalottchen/12998455244
- 5. Image courtesy of See-‐Ming Lee https://www.flickr.com/photos/seeminglee/4556156477
- 6. Reprioritizing the Relationship Between HCI Research and Practice: Bubble-Up and Trickle-Down Effects Colin M. Gray Indiana University Bloomington 201 N. Rose Avenue, #2276 Bloomington, IN 47405 USA comgray@indiana.edu Erik Stolterman Indiana University Bloomington 901 E. 10th St. Bloomington, IN 47405 USA estolter@indiana.edu Martin A. Siegel Indiana University Bloomington 901 E. 10th St. Bloomington, IN 47405 USA msiegel@indiana.edu ABSTRACT There has been an ongoing conversation about the role and relationship of theory and practice in the HCI community. This paper explores this relationship privileging a practice perspective through a tentative model, which describes a “bubble-up” of ideas from practice to inform research and theory development, and an accompanying “trickle-down” of theory into practice. Interviews were conducted with interaction designers, which included a description of their use of design methods in practice, and their knowledge and use of two common design methods—affinity diagramming and the concept of affordance. Based on these interviews, potential relationships between theory and practice are ex- plored through this model. Disseminating agents already common in HCI practice are addressed as possible mecha- nisms for the research community to understand practice more completely. Opportunities for future research, based on the use of the tentative model in a generative way, are considered. Author Keywords Interaction design; design practice; design methods; com- munity of practice. ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION In the field of HCI there has been an ongoing discussion about the relationship between academic and professional practice. The lack of recognition and adoption of research results among practitioners has been seen by some as a problem for academic research [10, 23, 26], while at the same time, practitioners have expressed that research is not addressing their everyday problems and therefore not offer- ing suitable tools for their needs [23, 27]. This study focus- es on how this current state of affairs impacts the practice community, and how researchers have the potential to in- form practice by first understanding it more fully. In this paper we will explore two aspects of the relationship between academic research and practice. We have labeled the two as the bubble-up effect and the trickle-down effect. The bubble-up effect describes the efforts of the practice community—and ideally the academic community as well—to refine and abstract situated knowledge and prac- tice of methods, tools, or concepts into refined theory and defined tools and methods. The trickle-down effect follows the more traditional research tradition, denoting and de- scribing the way adaptation of research and theory is com- monly seen to take place in design practice, including the opportunistic use of methods, tools, or concepts that origi- nate in an academic community. Even though the bubble-up and trickle-down phenomena indicate an exchange of expertise and knowledge between research and practice, it is often not a well-functioning rela- tionship where both sides strongly benefit from the other side, or even share core values about what constitutes prac- tical knowledge. This mismatch in the relationship between research and practice is problematic for several reasons: it means that researchers in many cases spend time and ener- gy devoted to research aimed at supporting practice that is largely ignored by practitioners; at the same time, profes- sional practice is often shaped by institutional and tradition- al norms and values that have the potential for improvement through research and practitioner partnerships. This mis- match can be understood as a general devaluing of the “oth- er side,” that is, researchers view practice as uninformed while practitioners view research as not being in touch with the reality of real world design. [27] argues that there is a need in HCI research to focus more on existing practice, understood on its own terms. In that study, the relationship of the designer to their tools in the act of designing was found to be significantly more complex than previously assumed. This intentional focus on practice revealed that conceptions of practice in academ- ia—which are then codified in theory and pedagogy—are frequently not grounded in any study of how designers practice “in the wild.” The practice community that re- searchers project does not necessarily represent the way HCI practice is actually conducted (Figure 1). This is not a new phenomenon; Rogers [23] reports on several earlier Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- tion on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy other- wise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. DIS 2014, June 21–25, 2014, Vancouver, BC, Canada. Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-2902-6/14/06..$15.00. http://dx.doi.org/10.1145/2598510.2598595 studies that show the same pattern where research results are not adopted by practice in the way intended by re- searchers. We see the mismatch, shown in Figure 1, as a serious problem for the HCI research community. Figure 1. Practice in situ and projected practice. There is even less research to be found in HCI addressing ways practice might positively influence research, that is, what we label the bubble-up effect, although practitioners have developed a wide range of professional conferences, blogs, and other tools to meet these needs internally (Figure 2). Figure 2. HCI research as processes of trickle-down and bubble-up This paper reports on our attempts to develop a better un- derstanding of the relationship between research and prac- tice, in particular from the perspective of practice and from practitioners. To explore some aspects of these phenomena we developed an interview study that documented general beliefs about and use of methods by design practitioners, and their knowledge and use of two specific methods— affordances and affinity diagramming—in their design practice. This research occurred in parallel with additional exploration of how researchers as developers of design methods conceive of their use in practice. We first discuss the existing relationship between academic research and practice as seen through the lens of the re- search tradition. We then discuss the history and context of two specific design methods, using this discussion as a framing to explore traditional conceptions of method use in practice and research traditions. Finally, we provide find- ings and discussion from interview participants in relation to these specific design methods, with implications for the relationship between academic research and practice. THE DYNAMIC RELATIONSHIP BETWEEN THEORY AND PRACTICE The connection between theory and practice is seen by many as vital, but often tenuous due to the intersection of competing communities of practice [6, 11, 16]. There are different ways to understand the relationship between these communities depending on focus and purpose. We have, in this project, worked with a tentative understanding of this relationship as the flow of information from one community of practice to another—along dimensions of appropriation [23] and abstraction of in situ detail. Rogers [23] traced the dimension of appropriation as it re- lates to the relationship between theory and practice, but only in the application of existing research to practice, that is, the appropriation of theory through use and adaptation in design practice. We will focus more strongly on the impli- cations of stimulating a flow from both sides, with equal representation and valuing of communities. The relationship between research and practice can refer to many different potential areas of concern, such as practical- ities (e.g., time, resources), competence, skills, and organi- zation [6]. In our research we focused on the way design methods are understood, developed, and used in the two communities of practice. We define design methods broad- ly, as any intellectual or practical support that a practitioner might use to support the design process in a positive way, encompassing everything from everyday methods, tech- niques, and tools for idea generation and collaboration, such as the "pen and paper," whiteboard, brainstorming, dialogu- ing, to methods and applications for sketching and refining ideas. For convenience, we label the flow of information about design methods from practice to research as bubble- up and the flow of information from research to practice as trickle-down (Figure 2). Based on this tentative model of the relationship between research and practice and a preliminary understanding of the two notions of bubble-up and trickle-down we devel- oped our research design. We first introduce our approach and the two design methods we decided to focus on. We then present our interview study and our findings. To con- clude, we return to our tentative model and our two con- cepts and how they can be further developed in a generative way to inform future research. OUR APPROACH As a way to examine how practitioners view design meth- ods, as defined in this paper, we decided to focus on two fairly well known design methods: affinity diagramming and affordances. We are well aware of the fact that these two methods are not necessarily methods in any strict sense; although affinity diagrams have been developed into more formal variations over time [2, 12] and understand- ings of affordances have evolved over time [13]. Our choice of these two methods were based on their dif- ference in apparent source, with affordances coming from ecological research methodology and affinity diagrams most often seen as coming from a business and marketing context (although the true source is cultural anthropology [1]). These methods also represent a range of method types: from generative and interpretative (affinity diagramming) to descriptive and analytic (affordances). PRACTICE IN SITU PROJECTED PRACTICE COMMUNITY ≠ PROJECTED PRACTICE COMMUNITY RESEARCH RESEARCH A. Trickle-Down Researchers create methods for use in practice B. Bubble-Up Researchers study of how practitioners use methods within the structures of the research community (if at all). A B We researched the history and background of each method, including: attention to the originating field or discipline, contributing authors or scholars, the first entrance of this method into HCI, and the current state of the method in HCI literature. We have no ambition to be comprehensive or to give a complete description of each method, since that is not the purpose of our work and we also expect most readers to be familiar with them. Instead, we provide some background information as a point of comparison from which interview data can be contrasted and viewed. AFFINITY DIAGRAMMING Affinity diagramming is commonly used as a tool in busi- ness to organize and cluster ideas or concepts. It appears to have been introduced to the Western world during the movement to Total Quality Control (TQC) in Japan, and was published as one of seven key business tools [28]. Alt- hough this tool became popular in a business context, it was originally known (and is still known in some Asian con- texts) as the KJ Method, which originates as a method for sorting ethnographic data. The KJ Method was created by Japanese cultural anthro- pologist Jiro Kawakita, and was originally created as a way to analyze data obtained through ethnographic methods in the 1960s [14]. Early examples of this method appear to be solitary without significant iteration, used as a way to “let the facts speak for themselves,” grouping facts based on relationships that were perceived to join them [1]. After individual relationships were established, groupings were narrowed down to a reasonable number, generally ten or less. While the method is still used this way in some con- texts, it is also used as a generative method, instigating the generation of data from stakeholders, and then evaluating relationships and connections between concepts, instead of only a way to evaluate a known data set. Beyer & Holtzblatt (2) are some of the first to discuss this method within the HCI community, introducing major con- cepts and procedures for carrying out this method from the original TQC books [28; see also 18, 30]. Very little re- search has been carried out specifically on the use or adap- tation of this method within the HCI context, aside from the creation of digital tools to support the core activities of the method [7]. However, various forms of Post-It note based diagrams seem to be common in both educational and pro- fessional settings. To what extent this use is true to the method of affinity diagrams is less known. AFFORDANCES Rogers [23] provides an excellent historical overview of the notion of affordance, from its early roots in ecological methodologies within the psychology community [8, 9, 25] to the importation of the key idea of affordances into the HCI community through the work of Don Norman [20]. The notion of affordances is defined by Norman as “…the perceived or actual properties of the thing, primarily those fundamental properties that determine just how the thing could possibly be used. […] Affordances provide strong clues to the operation of things. […] When affordances are taken advantage of, the user knows what to do just by look- ing: no picture, label, or instruction is required.” [20] However, today, there are many different definitions and the way the concept is used differs widely [13]. The under- lying concept of affordances is used both with and without its historical roots. In recent HCI scholarship, the notion of affordances has been re-examined. According to Kaptelinin & Nardi [13], the early work of Gibson informs the concept of affordanc- es, describing the notion of interaction between animals and the environment as a primary source for insights. They dis- cuss the potential for mediated action as a way to recast affordances as a method for investigating “distinctly human uses of interactive technologies.” [13]. This re-examination of the concept attracted substantial interest at the session where it was presented at CHI in 2012. According to Rogers [23] the notion of affordances is an example of a theoretical construct that has transitioned from research to practice while, in the process, losing its com- plexity and richness. Rogers notes that many practitioners recognize the concept and even “use” it without knowing its background or history. INTERVIEW STUDY The purpose of our interview study was to find out how the two design methods we selected—affinity diagrams and affordances—are currently understood and used by practi- tioners. We were particularly interested to discover how much practitioners know about the background and history of these methods, and where and how they learned about it, and to what extent they use it—that is, the trickle-down effect. It is important to note that this interview approach inherently defines and values these methods from a research perspective by using labels from that tradition; issues of labeling become clear in the analysis of the interview data. Implementation and use of a method in practice, regardless of knowledge of the history or background of a method would be sufficient to see it as an example of the trickle- down effect, rendering these aspects of the method suffi- cient, but not necessary. Similarly, the “core” of these methods may have also been developed and used by practi- tioners without knowledge of the “proper” title. This simi- larity between practice-originated and research-originated methods would not be discovered unless practice-centered research was carried out. We also wanted to find out if the practitioners were aware of any professional practices that have been developed and manifested in more formalized or structured ways or informed attempts to construct design methods based on practice and shared with a professional or research community—that is, the bubble-up effect. Data Collection We selected 13 practitioners from 12 companies, beginning with an opportunistic sample of HCI alumni from a large university, and then snowballing to include colleagues and other figures in their professional networks. All of the prac- titioners we located and subsequently interviewed worked as an interaction designer, user experience designer, or user researcher in some way. They had a range of professional experience: several practitioners had 10 or more years of experience, while the majority had worked 3-10 years. The practitioners had a variety of educational backgrounds: of the 13 interviewees, eight had an HCI-oriented educa- tion, three had a technical or engineering background, and one had a visual design background. Five of the interview- ees were female, while the remaining eight were male. A phone or face-to-face interview was scheduled with each participant. Each interview followed a semi-structured for- mat, and was audio recorded to facilitate further analysis. In addition, a dedicated researcher took detailed notes either during or shortly after the completion of the interview. A subset of the interview participants (Table 1) was select- ed based on numerous criteria to preserve the diversity of the original sample, including years of experience, voca- tional background, nationality, and gender. This subset is used to facilitate a more detailed discussion of our findings. Name (Pseudonym) Years of Experience Vocational Background Cheryl 10+ Business Abbie 20+ Computer Science Vicki 4 HCI Phil 2 HCI Jagdish 3 HCI Table 1: Subset of interview participants. Findings Our interviews covered a broad set of questions aimed at understanding professional practice in the field. In this par- ticular analysis we have focused on a small portion of the overall interview. These questions do not probe all aspects of the relationship between theory and practice, but they do provide us with some insights into the way practitioners think and reflect when it comes to design methods. Where did they learn methods? Some participants learned methods in graduate school, but in most cases dissemination of methods was through en- counters with colleagues in a design context. Phil specifi- cally noted his formal education in HCI as informing his knowledge and use of methods, while other designers who had practiced in the field without graduate work in HCI relied more on their colleagues, professional conferences, or publications for their understanding of design methods. Due to the relative newness of the field of interaction de- sign and the presence of many in the field who are trained primarily in other fields, a strong cross-pollination between related disciplines such as marketing, visual design, and business lead to the adaption and adoption of new methods. Cheryl experienced this phenomenon, tracing an important method for doing “guerilla research” that had been in- formed by her boss, who had learned this method of re- search from a marketing company. Cheryl has since written about related methods, further disseminating these ideas into her company and practice community. Participants sometimes felt that they had made a method up or felt that it was common sense approach, and then found a name to apply to that idea or concept later on. Abbie dis- cussed the “common-sense” nature of affinity diagramming as a basic way of filtering data. She had used this general method prior to learning about it in a more formal sense at an early CHI conference. What is their conception of research and theory in their prac- tice of design? While most of the practitioners we interviewed seemed aware of the research and academic community, they did not attempt to forge a connection between their community of practice and the academic community or professional academic community (e.g., ACM/CHI). Cheryl discussed her impression that CHI was generally populated by gradu- ate students, and that she believed in “spreading the wealth” by sending her employees to a broad range of more practi- tioner-focused conferences, looking to her employees to discover which events were most valuable. This shift to practitioner-focused events was a general trend from our interviewees, with a more direct emphasis on justifying the expenditure to attend these conferences through real and tangible gains in applicable methods. Abbie reflected on the role of professional conferences, especially CHI, noting that there was too much “big lan- guage” and not enough “beer and steak” interaction with peers. This shift to the theoretical and away from the practi- cal was a main reason she has shifted to other venues over time, as she describes herself as an “applied person.” Most interviewees seemed aware of theory, and how it might inform patterns of research or testing, but as practitioners, they mentioned the constraints of client budgets, and how the budgeting process often came in direct conflict with the “proper” application of theory or research-driven approach- es to design or user research. Cheryl discussed the frequent conflict between the need for research, and the lack of willingness from clients to pay for this research. As a result, her company has drifted from more traditional research methods to high-value, low-cost methods of user research, like guerilla research. How do they utilize methods in their design process? According to almost all of our participants, designers want methods that are easy to explain to their clients and easy to visualize, for the purpose of assisting them in communi- cating their design ideas. Ease of explanation and ease of visualization often seem to go hand-in-hand, suggesting that methods can be a powerful tool not only during the design process, but also to facilitate communication about design. Jagdish noted the use participatory design methods to quickly engage clients and designers around core issues, including them in key design decisions within the interface. In conjunction with engagement or communication, level of detail was a concern that Phil surfaced, as he related that the “state of finality” influenced the designers and clients he worked with, and thus preferred mediums and tools that didn’t focus on details (e.g. Photoshop, Illustrator). The practitioners seem to be highly opportunistic in their selection and use of methods, often selecting methods based on context, time and tools available, and designer knowledge. Phil used methods in a highly opportunistic way, selecting methods primarily based on the proximity of the individual: using a document or Microsoft PowerPoint for people who were unavailable, while preferring to use wireframes with markers for someone in the same office. In contrast, Abbie and Jagdish seemed to focus more tightly on one set of methods—mental models for Abbie and usa- bility testing and iteration for Jagdish—that they applied opportunistically, varying by fidelity and approach given the specific design context. Designers also adapt methods as needed for a given design situation, often seemingly unconsciously, to meet the con- straints or requirements of the specific design problem be- ing addressed. Vicki discussed her use of “scrappy” design methods, noting that many “textbook” methods required more time than she had to do “the right way,” so adapting methods was often the only way to carry out a given meth- od in practice. She further described the importance of im- plicitly understanding your audience, and using appropriate terms and methods to communicate with that audience. In particular, she described the differences between sharing information with other designers versus working with peo- ple from marketing or their human factors lab, highlighting the importance of speaking a consistent language with the people you are communicating with, “translating” termi- nology where appropriate between disciplines. What do they know about the original intent or historical context of the method? For other participants, conceptual knowledge of the ideas behind a method was most important, not knowledge of specific “traditional” methods or processes known by a certain name. The company Cheryl works for adapted affin- ity diagramming based on experiences of some of their de- signers, using more defined division of labor between de- signers (including moderation) and “throw[ing] the post-it note over their shoulder” after each idea is complete. This type of adaptation, either corporately or in a specific design situation, was quite common across all of our participants. Most importantly, the interviewees seemed to be focused on when to use design methods rather than basing their use on the origin or historically informed use of a method. Phil related an instance of a semantic differential, which was originally referred to as an affinity diagram in our inter- view, but was then retracted. Later in the conversation, Phil demonstrated a robust hybrid method that he and a col- league had developed, which included many ideas from affinity diagramming. It is clear that knowledge of the origin of a given method, or even prescribed steps to ac- complish that method, is not a barrier to using the concep- tual idea behind a method in a specific design context. Yes No N/A Affinity Diagramming Concept 10 2 1 5% Historical Context 0 12 1 6%Affordances Concept 10 1 2 3% Historical Context 3 8 2 3% Table 2: Participant knowledge of methods In Table 2, the participants’ knowledge of two selected methods is detailed. N/A refers to participants who were not explicitly asked, or did not explicitly answer, the question. While most participants were comfortable interacting with the general concept behind each method, confirmed by a discussion of core ideas from the methods in their design practice, their knowledge of the historical background or context of the method was more limited. Even in the case of affordances, where the participants knew some historical context surrounding affordances, their knowledge was lim- and detailed implementation of the original method, these designers also seemed to be more willing to adapt and ap- propriate these methods in practice. DISCUSSION In our discussion we will focus on the relation between research and practice that emerged from our interviews, primarily the notions of appropriation, abduction, and dis- seminating agents. We will then return to our tentative model and reflect on how our findings can be interpreted in the light of the model and the concepts of bubble-up and trickle-down. We will also add the notion of cycle-around and discuss the concept of design judgment. Through our analysis of the interviewee responses in rela- tion to our tentative model presented here (Figure 3), ap- propriation and abduction were identified as core activities in both directions. Concepts discussed in the text are linked to the model where relevant. We found that appropriation of research and theory (A) is frequent, especially in the application and context-of-use for specific methods by practitioners, resulting in abduction of research methods vis-à-vis real world contexts, while not necessarily in the form expected by those who develop new methods and tools. The appropriation is in many cases radi- cal. Sometimes only core aspects of a method are preserved and translated into actual practice. Appropriation is found in the intentional or unintentional reshaping of existing methods by individual practitioners. We found this appropriation to be an essential part of using methods, not necessarily a misuse of the original method. This alters the original meaning of what it means that a method is “used” in practice. When methods are radically appropriated in an abductive sense, even though they are originally highly formalized, they are still “used” and some- times emerge as an even richer method in situ. We also found that the lack of communication between researchers and practitioners, in the eyes of the interview- ees, leads to inaccurate and incomplete abstractions of practice (Figure 1). This perceived lack of understanding of design practice among researchers seems to be the core reason why the practitioners do not show any serious inter- est in investigating new methods developed in academia. Disseminating Agents Even though appropriation of academic results does not seem to be common among practitioners, it does happen. Based on the interviews we conducted, a number of dissem- inating agents were identified that commonly bring design methods into practice. While research and theory are often assumed to be connected—at least peripherally—to prac- tice, none of the practitioners we interviewed relied on any traditional academic sources for information regarding new methods or perspectives on tool use. Disseminating agents seemed instead to be linked to indus- try norms of employment, especially regarding which pools of people work in a given segment of the industry, with practitioners informing their own community in an educa- tive sense (as shown with E in Figure 3). For example, in more traditional graphic design or marketing fields (even if they are UX centered), prevailing norms of those industries tend to dominate. In newer, UX-focused teams, another balance of people and sources of information dominates. Regardless of industry segment, primary disseminating agents were people (e.g., coworkers and colleagues from professional groups and associations) and Internet discov- ery through blogs, news articles, Twitter, and trade publica- tions available online. Of particular interest is the role of coworkers in shaping or redefining a person’s feelings and knowledge about methods. Some of our interviewees learned methods from specific coworkers, or in the context of on-the-job training, while others learned methods more
- 8. Image courtesy of Marco Arment https://www.flickr.com/photos/jonwiley/3169246516
- 9. Based on image by Oliver Waters https://www.flickr.com/photos/owaters/3846053408 PRACTICE ACADEMIA
- 10. WHERE DO WE GO?
- 11. Image courtesy of Mary Madigan" https://www.flickr.com/photos/marypmadigan/9505064987 TRICKLE DOWN bringing emerging research into the practice community
- 12. 12Photo courtesy of Marcus Spiske https://www.flickr.com/photos/125167502@N02/
- 13. 13Photo courtesy of Muao https://www.flickr.com/photos/muao/5609676639/
- 14. Image courtesy of mullenkedheim https://www.flickr.com/photos/mullenkedheim/5101515763 BUBBLE UP Bringing traditions of practice into the research community
- 15. Image courtesy of cannedtuna https://www.flickr.com/photos/80296347@N00/4853379256
- 16. Image courtesy of mullenkedheim https://www.flickr.com/photos/mullenkedheim/5101515763 BUBBLE UP Image courtesy of Mary Madigan https://www.flickr.com/photos/marypmadigan/9505064987 TRICKLE DOWN
- 17. 17 Image courtesy of Sean Munson https://www.flickr.com/photos/83413148@N00
- 18. 18Goodman, E. (2013). Delivering Design: Performance and Materiality in Professional Interaction Design. (Unpublished doctoral dissertation). University of California, Berkeley, CA.
- 19. " CYCLE-‐AROUND
- 20. THANK YOU " uxdesignpractice.com NSF Grant #1115532 " colingray.me "