![10 Design for Usability Methods Tools
products [10], and to make matters even worse (for design
teams that is) an increasing pressure on time to market
[11]. The rapid development cycles put pressure on product
development activities: there is less time to perform usability
tests and the recommendations made by the users during
these tests cannot always be implemented [12]. In addition,
implementing market feedback based on an existing product
can be troublesome, because the development of a new
model starts directly after the previous product design has
been finalized. So, when the team starts working on the new
product, its predecessor often still has to be introduced on
the market [13].
Due to the increasing complexity of products, new product
development requires the collaboration of multidisciplinary
teams [14]. Product development teams are often distributed
across the planet [12, 15] which complicates team
communication [16]. Additionally, product development
groups often use local subcontractors or outsource
development activities which is considered detrimental for
product quality [2].
Usability under pressure
Many signals from the product development practice have
alerted us to the fact that the usability of electronic consumer
products is coming under pressure. In the past, product
returns and complaints were largely due to technical failures
(quality or reliability issues). Over time companies have
become better and better at managing product quality
and, up to the late nineties, this resulted in a decline in the
number of product returns [2]. However, from that time on
the number of product returns has been on the rise [13]. In
a study by Den Ouden et al. [10]no technical fault could be
detected in 48% of products returned by consumers. This ‘no-
fault-found’ category has been estimated at 68% of returned
electronic consumer products, and the cost for product
returns for 2007 in the US market alone was put at $13.8
Underlying trends
In this section, we review each of the drivers of the Design for
Usability project in more detail.
In electronic products
The number of functions in electronic products has been
steadily increasing for a number of years [1-3]; this is a result
of both the advances in technology [4, 5] and the commercial
advantage of offering innovative and increasing functionality
[6]. At the same time, many electronic products are getting
smaller and smaller [3] Products with a more elaborate
functionality are generally harder to use than those with
a limited number of functions [7, 8] and this becomes an
even greater problem when all these functions have to be
accessed through a small user interface [9].
In product development practice
Along with the increase in functionality, we have seen
an increase in the technological complexity of electronic
Figure 1: Increasing functionality, decreasing dimensions, and increased
networking results in a decrease in the ‘guessability’ of music players.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-12-320.jpg)
![11chapter 1 The Design for Usability project
Product impact: improve understanding of how users
change in the process of interaction with products,
and to integrate this knowledge in design practice, by
means of a Product Impact Tool.
Two additional topics were tackled by other members of the
project team:
Envisioning use: devising a technique which helps
teams create a common vision on product use at an
early phase of product development by team alignment
and sharing information with regards to future use
situations;
Dynamic and diverse use situations: gaining insights
into how designers deal with designing products that
are used in continually changing (dynamic) or different
(diverse) contexts of use, and translating these insights
into guidelines.
Participating parties
The Design for Usability project was Initiated and executed
by:
Delft University of Technology
Eindhoven University of Technology
University of Twente
In cooperation with and supported financially by:
Indes
Océ Technologies
Philips
T-Xchange
billion [17]. This return of products, even though technically
speaking they are not broken, can partly be attributed to
people not understanding how to use a product properly
and therefore concluding that it does not work, as well as
to those consumers dissatisfied with the product because it
did not meet their expectations [10]. Improving the usability
of products is seen as one of the strategies to deal with this
rise in returns [17].
The project
Goal
The Design for Usability project aims to improve the usability
of electronic professional and consumer products by
creating new methodologies and methods for user-centred
product development which can be quickly and easily
applied in design practice.
Work packages
Five primary themes formed the basis of the DfU project,
each of these themes was investigated as part of a PhD
project:
Design methodology: developing a project kick-off tool
that supports product development teams in specifying
a detailed user-centred plan of approach;
Barriers and enablers for usability in practice: identifying
which properties of product development organizations
influence usability;
Usability decisions in design practice: identifying critical
factors that influence the quality of usability-related
decision-making and that may cause teams to overlook
or misjudge usability problems;
User characteristics, product type, and soft usability
problems: identifying which combinations of personality
traits and product types lead to usability problems that
trigger user dissatisfaction;](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-13-320.jpg)
![12 Design for Usability Methods Tools
T-Xchange
Unilever
P5 consultants
Rianne Valkenburg
NL Agency
In addition to the team above, the project was made possible
by:
Elke Den Ouden: chair of the user committee
(before 2010)
Onno van der Veen: chair of the user committee
(from 2010)
Bart Ahsmann (TU Delft): consultant on valorisation issues
Approach
Integration of user-centred design
The Design for Usability project team took an integrated
approach to user-centred design (UCD). We studied product
development processes, teams and organizations, rather
than just investigating how user tests were conducted or how
usability departments are organized. We were interested in
discovering how the principles of user-centred design are
integrated in the product development process.
Practice-oriented
The aim of the Design for Usability research project,
improving how usability is dealt with in ‘real life’ product
development, necessitated a practice-oriented research
approach. At the same time as our initial exploration of the
topic by reviewing the literature, interviews were conducted
with usability practitioners and experts. Secondly, a major
part of data collection was achieved by working on case
studies [18] in product development practice. Thirdly,
many users were involved by means of survey studies
and experiments. Finally, the DfU project involved regular
‘member checks’ [19]: throughout the studies, specialists
verified any interpretations and conclusions, and each of the
Funded by:
The Netherlands Ministry of Economic Affairs, Agriculture and
Innovation through NL Agency under the IOP-IPCR program.
Team
The project Design for Usability was a collaboration between
a large group of design and development specialists.
The core research projects were performed by five PhD
candidates. Additional researchers from the three universities
were involved in the project, working on smaller but related
research projects.
PhD candidates
Steven Dorrestijn
Christelle Harkema
Frederik Hoolhorst
Cha Joong Kim
Jasper van Kuijk
Researchers
Stella Boess
Mieke van der Bijl-Brouwer
Henri Christiaans
Peter Sonnemans
Ilse Luyk-de Visser
Peter-Paul Verbeek
Mascha van der Voort
Tristan Weevers
Project coordination
Daan van Eijk (chairman)
Sonja van Grinsven
User committee
Indes
Océ
Philips](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-14-320.jpg)
![14 Design for Usability Methods Tools
v. d. Aalst, H. Corporaal, J. B. O. S. Martens, P. H. A. v.
d. Putten, A. J. M. M. Weijters, and A. C. Brombacher,
“Soft Reliability: An Interdisciplinary Approach with a
User–System Focus,” Quality and Reliability Engineering
International, vol. 25, pp. 3-20, 2008.
[5] D. A. Norman, The Design of Everyday Things. New York:
Basic Books Publishers, 2002.
[6] J. M. Thölke, E. J. Hultink, and H. S. J. Robben,
“Launching new product features: a multiple
case examination “ Journal of Product Innovation
Management, vol. Volume 18, pp. 3-14, 2001.
[7] J. Keijzers, E. den Ouden, and Y. Lu, “Usability
References
[1] D. A. Norman, “Three challenges for design,”
Interactions, vol. 14, pp. 46-47, 2007.
[2] E. Den Ouden, “Development of a design analysis
model for consumer complaints : revealing a new class
of quality failures “ PhD thesis, Eindhoven University of
Technology, Eindhoven, 2006.
[3] C. Lindholm, T. Keinonen, and H. Kiljander, Eds., Mobile
Usability. How Nokia changed the face of the mobile
phone. McGraw-Hill, 2003.
[4] A. Koca, M. Funk, E. Karapanos, A. Rozinat, W. M. P.
Figure 3: In the Design for Usability project, manuals, card sets, and booklets were used to disseminate knowledge, methods and tools to practitioners.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-16-320.jpg)
![15chapter 1 The Design for Usability project
Marketing Perspectives,” Journal of Product Innovation
Management, vol. 14, pp. 35-47, 1997.
[17] T. Steger, B. Sprague, and D. Douthit, “Big Trouble with
No Trouble Found: How Consumer Electronics Firms
Confront the High Cost of Customer Returns,” Accenture,
2007.
[18] R. K. Yin, Case study research: design and methods,
4th edition ed. Thousand Oaks, California, USA: SAGE
publications, 2009.
[19] A. K. Shenton, “Strategies for ensuring trustworthiness in
qualitative research projects,” Education for Information,
vol. 22, pp. 63-75, 2004.
benchmark study of commercially available smart
phones: cell phone type platform, PDA type platform
and PC type platform,” presented at 10th international
conference on Human computer interaction with mobile
devices and services, Amsterdam, The Netherlands,
2008.
[8] R. T. Rust, D. V. Thompson, and R. W. Hamilton,
“Defeating feature fatigue,” Harvard Business Review,
vol. 84, pp. 98-107, 2006.
[9] T. Keinonen, “One-dimensional usability - Influence of
usability on consumers’ product preference,” Smart
Products Research Group, University of Art and Design,
Helsinki, 1998.
[10] E. Den Ouden, L. Yuan, P. J. M. Sonnemans, and A. C.
Brombacher, “Quality and Reliability Problems from
a Consumer’s Perspective: an Increasing Problem
Overlooked by Businesses?,” Quality and Reliability
Engineering International, vol. 22, pp. 821-838, 2006.
[11] A. C. Brombacher, P. C. Sander, P. J. M. Sonnemans, and
J. L. Rouvroye, “Managing product reliability in business
processes ‘under pressure’,” Reliability Engineering and
System Safety, vol. 88, pp. 137–146, 2005.
[12] S. Minderhoud and P. Fraser, “Shifting paradigms of
product development in fast and dynamic markets,”
Reliability Engineering System Safety, vol. 88, pp. 127-
135, 2005.
[13] A. C. Brombacher, “Reliability in strongly innovative
products; a threat or a challenge?,” Reliability
Engineering System Safety, vol. 88, pp. 125-125, 2005.
[14] M. Kleinsmann, “Understanding Collaborative Design,”
PhD thesis, Delft University of Technology, Delft, 2006.
[15] P. Ketola, “Integrating Usability with Concurrent
Engineering in Mobile Phone Development,” PhD thesis,
University of Tampere, Tampere, Finland, 2002.
[16] X. M. Song, M. M. Montoya-Weiss, and J. B. Schmidt,
“Antecedents and Consequences of Cross-Functional
Cooperation: A Comparison of RD, Manufacturing, and](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-17-320.jpg)
![17
chapter 2
Interaction, experience, usability
and user-centred design
Learning from existing research
Jasper van Kuijk
The speed dial on this jigsaw incorporates knowledge that most users don’t
have, namely what the frequency of the saw blade should be for each material.
A frame of reference
Before presenting the methods and tools that were
developed within the Design for Usability project, we want
to provide a frame of reference. A great deal of theory and
method development in the field of user-centred design
has been accomplished previously, so we are not about
to let that go to waste. Secondly, having a shared view
on concepts like interaction, experience, and, of course,
usability, is essential for effective communication of the
results of the project. So below we discuss how we defined
electronic consumer products, and how usability, interaction
and experience relate. Lastly, we discuss the properties of the
approach that has the intention of creating usable products:
user-centred design.
Electronic products
The Design for Usability project focused on the usability
of electronic products for the professional and consumer
domain. These are products with a physical presence (as
opposed to software), and that feature a certain amount
of integrated information technology that enables them to
interact with the user (as opposed to, e.g., chairs and vases).
Some examples are mobile phones, microwaves, office
printers and MRI scanners.
Differences with the digital domain
Important differences between electronic products and the
digital-only domain (such as ICT systems, websites and
software) are:
Physical presence: the usability of electronic products is
not only influenced by the on-screen user interface, but
also by the design of other input and output modalities,
and the embodiment of the product;
Unique platforms: digital products usually run on more
standardized technological platforms (servers, operating
systems, motherboards, etc.) than electronic products, for
which often custom-built hardware has to be developed;
Design/development characteristics: when developing
software it is more common to have design and
implementation coincide; the product is designed while
being programmed [1]. Secondly, in the development](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-19-320.jpg)
![18 Design for Usability Methods Tools
of digital products there is no clear moment when the
design is more or less ‘frozen’ because investments
have been made in moulds and orders for parts have
been placed.
Ecosystems
Electronic products increasingly function as a part of
a network of products and services, referred to as the
‘ecosystem’ [1-3] (Figure 1). For example a television can
be part of a ‘network’ with a home cinema set, a hard disk
recorder and a satellite decoder. Even if the usability of the
individual products is acceptable, this does not guarantee
that the system as a whole is usable. In product ecosystems
three sub-categories can be distinguished [4]:
1 The core product: what the user primarily interacts with;
2 The extended product: those parts that facilitate the use
of the core product;
3 Symbiotic products, software, services and content that
allows the core product to function.
User interface
The user interface (UI) consists of those parts of the product
which enable users to use it for its intended purpose [5].
It includes all elements of a product-service system that
allow people to control it and receive information on how
to interact with it. Examples of elements of the UI are, for
example, the display and on-screen UI, controls and audio
output components). But also the embodiment of a product
can be considered part of the user interface, as users may
also attribute a certain meaning to the appearance of the
product, for example which product category it falls into
and how it (thus) should be operated [6], and, because the
embodiment influences how the product can be interacted
with physically, the physical (dis)comfort users experience
[7].
What is usability?
Most of us have at one time faced sales people who tell
us, ‘This product does this and that’. This is fine, but what
is most relevant is what you can do with the product. As a
common mantra in user-centred design goes, ‘If the user
can’t find it, it’s not there.’ When talking about usability,
we take an instrumental view on products. We are not
interested in whether you want to buy them, own them
or caress them: we are focused on whether you can use
Figure 1: Visualization of the primary components of electronic consumer products, clustered by the core product, extended product and symbiotic components.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-20-320.jpg)
![19chapter 2 Interaction, experience, usability and user-centred design
them. In other words, whether you at least can do what its
maker intended you to do with the product, and maybe more.
Defining usability
The most commonly used definition of usability is the one by
the ISO organization [8]:
“The extent to which a product can be used by
specified users to achieve specified goals with
effectiveness, efficiency and satisfaction in a
specified context of use.”
In which the three main dimensions of usability are defined as:
Effectiveness: accuracy and completeness with which
users achieve specified goals.
Efficiency: resources expended in relation to the accuracy
towards the use of the product. E.g., required time or
mental effort.
Satisfaction: freedom from discomfort, and positive
attitudes towards the use of the product.
These three dimensions are quite generic. They define what
makes a product usable at a conceptual level: whether users
can reach their goals, at what cost, and how they feel about
using the product. However, these dimensions are not very
specific. Depending on the (type of) product, it still needs to
be specified how to operationalize them, which measures
can be used, and what the essential goals per phase of use
are [9]. In different phases of product use, different ‘types’ of
usability can be important, e.g., guessability, learnability or
experienced user performance (Figure 2).
Usability and functionality
The goals that a product can help the user achieve, such
as cleaning clothes or playing music, are referred to as its
functionality or utility [10, 11]. The ISO definition of usability
refers to the extent to which users can apply the product to
reach their goals. This implies that usability is not only about
whether people can access the functionality the product
offers, but also about whether the product offers the right
functionality. Take for example the usability of E-books versus
the usability of conventional books. For the goal of quickly
looking up a page and scanning the content, conventional
books can be considered to be more efficient, whereas for
the goal of being able to read during your holiday, an eBook
may be a more effective and efficient solution.
Time
per
task
Experience of task
(task expertise)
Relearnability
Gulf
System potential
EUP (Experienced User Performance)
Learnability
Gulf
Guessability
Gulf
Figure 2: Performance curve with components
of usability as specified by Jordan [1]. The
time required to complete tasks is displayed
as a function of how long users have used
the product. First people have to familiarize
themselves with the product and learn how
to operate it (‘guessability gulf’), followed
by a first phase of use during which people
learn how to use the product (‘learnability
gulf’). The ‘relearnability gulf’ refers to how
repeat use during time per task is influenced
by the ease with which product use can be
recalled. And finally, as users gain more and
more experience, time per task converges on
‘experienced user performance’.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-21-320.jpg)
![20 Design for Usability Methods Tools
Human-product interaction is a complex interplay of the
components of the context of use: the users, tasks, product,
and its environment [8]. Based on existing frameworks [5,
10] within the DfU project, we developed a new framework
for human-product interaction specifically with electronic
products in mind [4]. It consists of the following elements:
Product-service combination: what the user applies to
reach a goal;
Symbiotic products: to stress that electronic consumer
products increasingly function in networks of products;
User(s): who interact(s) with the product. Includes
the goals (intended outcomes) they have, as well
as characteristics like knowledge, skill, experience,
education, training, physical attributes, and motor and
sensory capabilities;
Other people: people who do not directly interact with
the product-service system, but who can affect and be
Human-product interaction
From the ISO definition it can be deduced that usability
includes both user performance (effectiveness and
efficiency) and user experience (satisfaction about use). User
performance can be observed or measured by looking at
human-product interaction, which is defined as [5]:
Bi-directional information exchange between
users and equipment, which may include
physical actions, resulting in sensory feedback.
We can distinguish between using a product and interacting
with it. Using a product is the application of a product
in order to reach a goal [12] - for example, we can use
a vase to hold water and flowers. Interacting with the
aforementioned vase would be for example filling it with
water and putting the flowers in it.
Figure 4: Framework for human-product interaction.
Figure 3: Left: Using a vase for the purpose of holding flowers and
providing them with water. Right: interacting with a vase with the purpose of
putting water in the vase.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-22-320.jpg)
![21chapter 2 Interaction, experience, usability and user-centred design
affected by the use of a product;
Environment: includes the physical environment (e.g.
workplace, furniture), the ambient environment (e.g.
temperature, humidity) and the social and cultural
environment (e.g. work practices, organizational structure
and attitudes).
Context-dependent
The properties of components of human-product interaction
can vary considerably. For example, the interaction that takes
place with a phone when a teenager is running to catch
a train while making a phone call is completely different
than that of a 45-year old man sitting at a kitchen table and
sending a text message. Human-product interaction, and
as a consequence usability, is determined by who uses a
product, what they are trying to achieve, the other products
in the network, other people present, and the environment in
which the product is used. Usability is defined as: ‘The extent
to which a product can be used by specified users to achieve
specified goals (…) in a specified context of use.’ In other
words: there is no such thing as the usability of a product, as
usability depends on the context of use [8].
User diversity
User characteristics that can influence human-product
interaction include knowledge, skill, experience, education,
training, physical attributes, and motor and sensory
capabilities[8]. These can vary greatly from one user to
the next and thus usability can vary greatly between user
groups. But even when observing just one person, we may
observe great differences from one day – or even from one
moment – to the next; even though bodily measures and
cognitive capabilities remain the same, the patterns of use of
individual users can vary greatly [13, 14].
Diversity in user goals
An important aspect which determines the usability of a
Figure 6: The relation between human-product interaction, user experience
and the ISO definition of usability. Two out of three dimensions of usability
(effectiveness and efficiency) relate to human-product interaction. The third
(satisfaction) is about how users experience that interaction (user experience).
Figure 5: Visualization of the relation between human-product interaction
(black, left) and the user experience (right, red). The user experience is the
user’s affective response to the interaction, which can iteratively influence the
interaction.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-23-320.jpg)
![22 Design for Usability Methods Tools
Usability: user performance and
user experience
If we look at the dimensions of usability, we can distinguish
between ‘user performance’ measures (effectiveness and
efficiency) and subjective assessments of ease of use
(satisfaction) [10]. We can state that the first two measures
refer to human-product interaction (what happens) and
the third refers to user experience aspects (how people
experience what happens).
Attention needs to be paid to both aspects, because the
relation between the two cannot always be predicted. In
some cases the user performance dimensions can be more
important, while in other cases user experience aspects
should be prioritized. Because satisfaction is influenced by
expectations [23] it could, for example, be that a person who
owns a particularly unusable MP3 player is very satisfied
about the effectiveness and efficiency of this MP3 player, and
that others would consider this to be mediocre, because they
previously owned a very usable model.
User-centred design
Creating products and product-service systems with a high
level of usability can be achieved by applying a user-centred
design approach.
The designer-user gap
Because users and situations in which products are used
can vary greatly, as a designer it is hard to predict all the
situations that users may use a product in. First of all,
because the designer may differ greatly from the person
he or she is designing for. Nielsen [24] refers to this as the
‘designer-user gap’. Secondly, the people who worked on the
product know much more about the product than the people
who will be using it, because they – after all – designed it,
often from the ground up. So they know all the intricacies,
all the menus, all the small tricks, that there is a button at
product is the goal that the user is trying to reach at that
time. The goal may vary with the different stages of product
use [15, 16]; users have a different goal when they are trying
to install a product, when first using a product, or when
using a product for the millionth time. User goals are also
influenced by the role of the user. The user of a system can
be to the so-called ‘end-user’, but he or she can also have
other roles. For example, for an office printer and copier, user
groups include the ‘end-users’ (who get coffee), service-
people (who refill the cartridges), and maintenance staff
(who install and maintain the product) [17]. But even within
the end-user group there can be a considerable variety of
user types. For example, a department secretary will use a
printer/copier in a different way than an office worker.
User Experience
The term ‘user experience’ has been widely used to describe
what using a product ‘does’ to the user. It is often billed as
the ‘next step beyond usability’; products should be usable,
but they should also appeal to people’s sense of aesthetics,
provide pleasure, etc. To define user experience we have
adapted a definition made by Forlizzi Battarbee [18]:
The constant stream of ‘self-talk’ that happens
as a consequence of being exposed to,
applying and interacting with products; how
we constantly assess our goals relative to the
products we observe and use.
Interaction is what happens, user experience is how
that makes us feel (Figure 4). The fact that it includes an
evaluative act seems to make user experience related to
the notion of satisfaction: whether a product lives up to the
expectations that people have about a product [19, 20].
This implies that the user experience can also be influenced
by factors such as advertising, word-of-mouth or product
appearance [21, 22].](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-24-320.jpg)
![23chapter 2 Interaction, experience, usability and user-centred design
the back you can push, that you need to connect this cable
before that cable, etc. It is difficult to ‘unlearn’ all that.
Principles of user-centred design
User-centred design helps to compensate for these issues:
the designer-user gap and the design team’s ‘unlearnable’
knowledge about the product’s workings. User-centred
design can be described as [25]:
“An approach in which ‘product quality should
be measured from a user point of view, taking
into account needs, wishes, characteristics
and abilities of the projected user group”
The ISO standard on Human-centred1
Design for Interactive
Systems [26] provides six basic principles:
1 The design is based upon an explicit understanding of
users, tasks and environments;
2 Users are involved throughout design and development;
3 The design is driven and refined by user-centred
evaluation;
4 The process is iterative;
5 The design addresses the whole user experience;
6 The design team includes multidisciplinary skills and
perspectives.
User involvement and representation
Basically, user-centred design brings an extra perspective
to the development process: that a product does not work
if users do not understand how it works. To make sure that
users understand the product, designers need to bridge the
designer-user gap by adding user-involvement to design.
1 Some prefer the term ‘human-centered design’ as they believe this does
justice to view that people are more than users of products.
User involvement is the act of collecting information about
users or from users [27, 28]. In some cases, conducting a
simulation with actual users is not possible, but product
developers still need to anticipate product usage, in which
case they might resort to so-called ‘inspection methods’ [29],
such as an expert evaluation or a method that methodically
simulates product use. As a counterpart to user involvement,
we refer to these methods as ‘user representation’](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-25-320.jpg)
![24 Design for Usability Methods Tools
[10] B. Shackel, “The concept of usability,” in Visual Display
Terminals, J. Bennet., Ed., ed Englewood Cliffs, New
Jersey: Prentice-Hall, Inc., 1984, pp. 45-81.
[11] J. Grudin, “Utility and usability: research issues and
development contexts,” Interacting with Computers,
vol. 4, pp. 209-217, 1992/8 1992.
[12] N. F. M. Roozenburg and J. Eekels, Product Design:
Fundamentals and Methods. Chichester: John Wiley
Sons, 1995.
[13] H. Kanis, “Usage centred research for everyday
product design,” Applied Ergonomics, vol. 29, pp. 75-
82, 1998/2 1998.
[14] M. Weegels, “Accidents Involving Consumer Products,”
PhD thesis, Delft University of Technology, Delft, 1996.
[15] P. W. Jordan, “What is usability?,” presented at
Contemporary Ergonomics, 1994.
[16] N. F. M. Roozenburg and J. Eekels, Produktontwerpen,
structuur en methoden. Utrecht: Uitgevery Lemma,
1991, 1991.
[17] J. Nielsen, “The usability engineering life cycle,” IEEE
Computer, vol. 25, pp. 12-22, 1992.
[18] J. Forlizzi and K. Battarbee, “Understanding
experience in interactive systems,” in 2004 conference
on Designing interactive systems: processes,
practices, methods, and techniques, Vienna, Austria,
2004, pp. 261-268.
[19] K. Matzler, H. H. Hinterhuber, F. Bailom, and E.
Sauerwein, “How to delight your customers,” Journal
of Product and Brand Management, vol. 5, pp. 6-18,
February 1996 1996.
[20] F. F. Reichheld, “The one number you need to grow,”
Harvard Business Review, vol. 81, pp. 46-55, Dec
2003.
[21] S. L. Wood and C. P. Moreau, “From Fear to Loathing?
How Emotion Influences the Evaluation and Early Use
of Innovations.,” Journal of Marketing, vol. 70, pp. 44-
57, 2006.
References
[1] B. Buxton, Sketching User Experiences: Getting the
Design Right and the Right Design: Morgan Kaufmann,
2007.
[2] I. M. De Visser, “Analyzing user perceived failure
severity in consumer electronics products :
incorporating the user perspective into the
development process “ PhD Thesis, TU Eindhoven,
Eindhoven, 2008.
[3] E. L.-C. Law, V. Roto, M. Hassenzahl, A. P. O. S.
Vermeeren, and J. Kort, “Understanding, scoping
and defining user experience: a survey approach,”
presented at the 27th international conference on
Human factors in computing systems, Boston, MA,
USA, 2009.
[4] J. I. van Kuijk, “Managing Product Usability; How
companies deal with usability in the development
of electronic consumer products,” PhD thesis, Delft
University of Technology, Delft, 2010.
[5] ISO, “ISO 20282-3: Ease of operation of everyday
products -- Part 3: Test method for consumer
products,” International Organization for
Standardization, Geneva, Switzerland2007.
[6] S. Boess and H. Kanis, “Meaning in product use: A
design perspective,” in Product Experience, H. N. J.
Schifferstein and P. Hekkert, Eds., Reprint Edition, 2009
ed Oxford, UK: Elsevier, 2008.
[7] P. Vink, Ed., Comfort and Design: principles and good
practice. Boca Raton, Florida, USA: CRC Press, 2005.
[8] ISO, “ISO 9241-11 Ergonomic requirements for office
work with visual display terminals (VDTs) - Part 11:
Guidance on usability,” International Organization
for Standardization, Geneva, Switzerland ISO 9241-
11:1998(E), 1998.
[9] N. Bevan and M. Macleod, “Usability measurement in
context,” Behaviour Information Technology, vol. 13,
pp. 132-245, 1994.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-26-320.jpg)
![25chapter 2 Interaction, experience, usability and user-centred design
[22] E. Karapanos, J. Zimmerman, J. Forlizzi, and J.-B.
Martens, “User Experience Over Time: An Initial
Framework,” presented at the CHI 2009, Boston,
Massachusetts, USA, 2009.
[23] G. Lindgaard and C. Dudek, “What is this evasive
beast we call user satisfaction?,” Interacting with
Computers, vol. 15, pp. 429-452, 2003/6 2003.
[24] J. Nielsen. (2008, May 30). Bridging the Designer-
User Gap. Available: http://www.useit.com/alertbox/
designer-user-differences.html
[25] K. Vredenburg, S. Isensee, and C. Righi, User-Centered
Design; An Integrated Approach. New Jersey, USA:
Prentice-Hall, 2002.
[26] ISO, “ISO 9241 Ergonomics of human-system
interaction,” in Part 210: Human-centred design for
interactive systems (formerly known as 13407). ed.
Switzerland: ISO, 2010.
[27] J. Grudin, “Obstacles to user involvement in software
product development, with implications for CSCW,”
International Journal of Man-Machine Studies, pp.
435-452, 1991.
[28] S. Lauesen, “Usability Engineering in Industrial
Practice,” in INTERACT’97, Discovering New worlds of
HCI, Sydney, Australia, 1997, pp. 15-22.
[29] T. Rooden, “Design models for anticipating future
usage,” PhD thesis, Applied Ergonomics and Design,
Delft University of Technology, Delft, 2001.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-27-320.jpg)
![30 Design for Usability Methods Tools
return the queue is gone. I use the ATM and withdraw 300
Euros. The ATM returns my card, but no money comes out.
Now I start worrying as well. I decide to try again, assuming,
hoping that the bank will not process the transaction if no
money came out. As I look down to read the instructions,
this time I notice the text at the very bottom of the opening
screen (that disappears as soon as you slide your card into
the slot): ‘This machine will not distribute sums greater than
250 Euros.’
People worried. People got annoyed. A queue formed at
the machine and inside the bank. I think its fair to say that
the customer experience of this bank suffered. But what
this situation made me wonder most is: why is this ATM so
unclear? Why just a statement on the opening screen? Why
not a clear warning if a transaction cannot be processed.
Why not simply not allow people to ask for an amount higher
than 250 Euro. Basically, it makes me wonder: what went
wrong during development. Did the designers not know
about interaction design principles? Was this use case not
involved in user testing? Was any user testing performed, for
that matter?
These are the type questions that I answered when working
on my PhD. The aim of my PhD project was to identify
barriers and enablers for usability in product development
practice of electronic consumer products ( the story above
about the ATM is an example from professional electronics).
The focus was on electronic consumer products (e.g.,
portable music players, washing machines and mobile
phones), because in this sector usability is under pressure
from increasing product complexity, commoditization and
speed of development.
In a discussion of research in the medical sciences, Malterud
[1] argues that in addition to controlled experiments, with
their focus on questions and phenomena that can be
Introduction
A couple in their late fifties is blocking the only ATM machine
in Budapest that spits out Euros. And I need Euros. The
couple turns out to be Australian. The man tried to withdraw
500 euros from the ATM, got his card back, but no money.
In a display of utter innocence the machine has returned to
its default state, showing just the welcome screen. The man
says he will go into the bank and ask what the couple should
do. The couple decides the woman will stay at the ATM in
case the money all of a sudden does turn up, and to prevent
other people from using the ATM, because they don’t want
to change the state of the machine. Perhaps the people from
the bank can find a way to get their money out. A queue
starts forming. Embarrassed, the woman tells the people in
line what happened, and why they cannot use the machine.
As an alternative people start walking into the bank, where
the Australian man is still waiting for his number to be called.
More and more people come into the bank, hoping to get
Euros at a cash desk. I leave. When, half an hour later, I
Author Bio
Jasper van Kuijk is Assistant
Professor at TU Delft with a
focus on the practice of user-
centred innovation. He obtained
a Master’s degree in Industrial
Design Engineering (IDE) at TU
Delft, specializing in human- product interaction of
electronic consumer products. He then worked two
years as a design management consultant before
starting his PhD in the DfU project.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-32-320.jpg)
![313.1 Organizational barriers and enablers for usability
controlled, measured and counted, the knowledge of
experienced practitioners should be studied, because this
could offer a broader understanding of a phenomenon. The
same is true for product development. Because experienced
product developers have been immersed in product
development on a daily basis, they can possess a wealth of
knowledge on what does and does not work for usability in
product development practice. This research project was set
up to tap into that knowledge.
Research method
In a case study on human-centred design (HCD) in the
development of ICT systems, Steen [2] observed that
human-centred design practice is very different from HCD
principles and theory. Having worked both in commercial
and academic environments, Dennis Wixon [3] and Donald
Norman [4] underline the contrast between usability
in research and in practice: real, day-to-day product
development is messy (at best), and that not only should
the workings of methods for user-centred design when
they are applied in a controlled setting be studied, but also
what happens when they are used in the fast-paced, hectic
process of developing electronic consumer products.
This research project consisted of three case studies. The
case study is a suitable method for explanatory studies into
‘a contemporary set of events over which the investigator
has little or no control’ [5]. The first study was interview-
based, and its goal was to explore how usability is dealt
with in four sectors ‘adjacent’ to the electronic consumer
products market (e.g., automotive, office coffee machines).
Secondly, an interview-based case study was conducted at
five major international product development groups in the
electronic consumer products sector. The goal of this study
was to identify barriers and enablers for usability in practice.
The third and final case study investigated the development
history of three electronic consumer products within one
product development group. This resulted in a detailed
Case study I
4 markets, 4 companies
Fast
Moving
Consumer
Goods
Office
Coffee
Machines
Office
Copiers
Automotive
Case study II
CE market, 5 companies
Fast
Moving
Consumer
Goods
Office
Coffee
Machines
Office
Copiers
Automotive
Case study III
One company, 3 projects
conceptual framework
conclusions, discussion
recommendations
recommendations
for industry
dissemination
literature
survey
exploratory
interviews
Fast
Moving
Consumer
Goods
Office
Coffee
Machines
Office
Copiers
Automotive
Electronic
Consumer
Products
Figure 1: The setup of this research project](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-33-320.jpg)
![34 Design for Usability Methods Tools
design, usability specialist, product manager, upper
management) from one company. Before the workshop,
each participant reads the card set and selects their three
most and least favourite recommendations and provides a
motivation for selecting them.
During the workshop multiple discussion rounds are held
according to the following steps:
1 One participant introduces a most (or least) relevant
recommendation and provides motivation in max.
3 minutes [facilitator probes for concrete examples,
argumentation, perceived benefits, potential roadblocks].
2 The facilitator asks whether other group members have
chosen the same recommendation (as favourite or
non-favourite). If so: provide opportunity for additional
motivation (also max. 3 minutes).
3 Rest of the group responds from their perspective (max
1. minute pp.). The facilitator probes for agreement,
disagreement, motivations, roadblocks, and benefits.
4 Move on to next participant, next recommendation.
At the end of the workshop the group tries to come to a
consensus on what they believe to be the three most relevant
and three least relevant recommendations. If the workshop
consisted of multiple groups, this selection is presented at a
plenary session, together with the argumentation.
Benefits
The case studies provide design researchers with a situation
comparable to case studies they might conduct, and the
insights they need to develop ‘designer-centred’ tools and
methods for user-centred design. The recommendations for
usability in practice provide actionable information on how
to setup a user-centred product development organization.
Conducting a workshop with the card set can be a starting
point for an action plan towards a more user-centred
organization. It brings to light the differences in viewsFigure 5: Explanation of the layout of the cards](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-36-320.jpg)
![36 Design for Usability Methods Tools
More information
Card set
The card set: Recommendations for Usability in Practice
http://bit.ly/be-cardset
rdset
PhD thesis
Managing Product Usability: How companies deal with
usability in the development of electronic consumer products
http://bit.ly/be-thesis
Product usability weblog
Weblog kept by Jasper van Kuijk during his PhD project
www.uselog.com
Response to recommendation #5:
‘Team: One roof: all disciplines - in one room - throughout the
process’
Design researcher at strategic design consultancy
“As a design researcher at [Strategic Design
Consultancy], used to working in multidisciplinary
teams, I very much agree with the statements you are
making above. Both in my work with project teams
and with clients, I have found that on top of this, there
is an important distinction between ‘meeting’ and
‘working’ together. Even if being situated full time in the
same space is not an option, there is a lot to gain by
organizing work sessions when you get the chance,
as opposed to meetings. The difference in my mind is
that the latter is focused on sharing information, talking
through issues and challenges; the former is focused
on working through design challenges together, as a
(multidisciplinary) team. In my experience this results in a
mind-set that is much more positive, focused on solutions
and understanding each other’s perspective, instead
of underlining problems, disagreements and company
politics.”
Based on the input received, the recommendations were
improved and a second round of feedback was collected.
The recommendation card set was published together
with the author’s PhD thesis, and can be ordered and
downloaded through the Design for Usability website, and
the author’s weblog.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-38-320.jpg)
![373.1 Organizational barriers and enablers for usability
References
[1] K. Malterud, “The art and science of clinical knowledge:
evidence beyond measures and numbers,” The Lancet,
vol. 358, pp. 397-400, 2001.
[2] M. Steen, “The fragility of human-centred design,”
PhD Thesis, Delft University of Technology, Delft, The
Netherlands, 2008.
[3] D. Wixon, “Evaluating usability methods - why the
current literature fails the practitioner,” Interactions, pp.
28 - 34, 2003.
[4] D. Norman, “Design as Practiced,” in Bringing Design to
Software, T. Winograd, Ed., New York, USA: ACM Press,
1996.
[5] R. K. Yin, Case study research: design and methods,
4th edition ed. Thousand Oaks, California, USA: SAGE
publications, Inc., 2009.
Author homepage
Homepage of Jasper van Kuijk, highlighting education and
research activities
http://bit.ly/vankuijk
Contact
For more information about this research project, feel free to
contact the author:
Jasper van Kuijk
Delft University of Technology
Industrial Design Engineering
+31 (0)15 278 1378
j.i.vankuijk@tudelft.nl
Core publications
Van Kuijk, J.I., H.C.C.M. Christiaans, H.Kanis, D.J. van
Eijk (accepted with revisions) ‘Barriers and enablers for
usability in practice – A multiple case study of product
development groups of electronic consumer products’
Van Kuijk, Jasper, L. van Driel, D.J. van Eijk (accepted
with revisions) Usability in practice; a cross-market
comparison. A multiple case study exploring usability in
product development practice in four markets’
Van Kuijk, J.I. (2010). Managing Product Usability; How
companies deal with usability in the development of
electronic consumer products. PhD thesis, Faculty of
Industrial Design Engineering. Delft, Delft University of
Technology.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-39-320.jpg)
![473.2 The UCD Kick-Off Tool
References
[1] J.I. van Kuijk. “Managing product usability: How
companies deal with usability in the development of
electronic products.” PhD thesis, Delft University of
Technology, 2010.
[2] P.H. den Ouden. “Development of a design analysis
model for consumer complaints: Revealing a new class
of quality failures” PhD Thesis, Eindhoven University of
Technology, 2006.
[3] J. Daalhuizen, P. Badke-Schaub, J. Fokker. “Community
Based Design Support” I. Horváth Z. Rusak (Eds), in
Proceedings of TMCE 2008 conference, Izmir, Turkey,
2008.
[4] N. Cross. Designerly ways of knowing. Basel, Boston,
Berlin: Birkhäuser, 2006.
[5] N. Cross, H. Christiaans, K. Dorst. Analysing design
activity. Chichester: John Wiley and Sons Ltd, 1996.
[6] Kerzner, H. Project management: A systems approach to
planning, scheduling and controlling. Berea, Ohio USA,
John Wiley Sons, 2001.
[7] PMI. A guide to the project management body
of knowledge (PMBOK Guide). Newtown Square,
Pennsylvania USA, PMI, 2001.
[8] Dorst, K. “Design research: a revolution-waiting-to-
happen.” Design studies 29: 4-11, 2008.
Contact
If you would like to know more about this research or would
like to experience a workshop with the UCD Kick-off Tool first
hand, feel free to contact the authors:
Frederik Hoolhorst
Industrial Design Engineering
University of Twente
+31 (0)53 4893 192
f.w.b.hoolhorst@ctw.utwente.nl
Mascha van der Voort
Industrial Design Engineering
University of Twente
+31 (0)53 489 2541
m.c.vandervoort@ctw.utwente.nl
Core publications
Hoolhorst, F. W. B. and M. C. van der Voort (2009). A
concept for a usability focused design method. IASDR
2009. Seoul.
Hoolhorst, F. W. B. (2010). Research project ‘Insight in
design practice’: Insight into the user-centred design
method’s applicability in design practice. Enschede,
Universiteit Twente.
Hoolhorst, F. W. B. (2012) Structuring user-centred product
development processes, Engineering Technology,
University of Twente, PhD thesis.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-49-320.jpg)
![50 Design for Usability Methods Tools
Introduction
A crucial challenge within User-Centred Design (UCD) is
getting users involved for example for obtaining information
about their needs or evaluating concepts. Hundreds of
UCD methods, tools and techniques have been developed
with the goal of helping product developers with this
challenge. These methods often serve specific needs such
as evaluating rough, conceptual, designs of interfaces or
learning about the behaviour and movement of users in
environments.
The product development practice can be extremely hectic
and messy [1], leaving practitioners with very little time
to explore and find a method or tool that matches their
situation. UCD methods are spread over many different
sources like scientific publications, books and online
collections (see Results / Overview of existing UCD method
collections). A number of attempts to collect these methods
and tools have only resolved part of the issue: they
provide an overview, but the number of methods is often
limited and the resulting list or categories do not match a
practitioner’s needs and preferences [2, 3]. Moreover, online
method collections provide little or ineffective guidance for
selecting the appropriate method [4, 5] and the descriptions
of methods often lack the practical information needed to
execute them [3]. Most information is given textually and
without practical guidance, instructions, or examples [6],
which (among others) can lead to a continued misuse of
common methods such as focus groups and observational
studies [7, 8].
The consequence of this inaccessibility and poor
documentation of method descriptions, is that designers
often stick to what they know, leaving many potentially
beneficial methods unused [9, 10]. This hinders the
development of the field and – eventually – the
development of better products. It is also a barrier for the
Authors Bios
Tristan Weevers holds a Bachelor’s
degree in Industrial Design from
the Saxion University of Applied
Sciences where he specialized
in User-Centred Design. He then
was awarded a Master’s in Design
for Interaction at Delft University
of Technology. His work was
rewarded with presentations at the Intel Developers
Forum (2009) and as a winner of the Design Challenge
(2010). Tristan developed the UCD toolbox during his
graduation project at TU Delft after which both parties
agreed to turn the project into a startup.
Tristan Weevers’ graduation project was supervised by
Assistant Professor Jasper van Kuijk, PhD candidate
Jaap Daalhuizen and Professor Daan van Eijk.
Jasper van Kuijk is Assistant
Professor at TU Delft with a focus
on the practice of user-centred
innovation. He obtained a Master’s
degree in Industrial Design
Engineering (IDE) at TU Delft,
specializing in human- product
interaction of electronic consumer
products. He then worked two years as a design
management consultant before starting his PhD in the
DfU project.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-52-320.jpg)
![52 Design for Usability Methods Tools
categorizations are often not in line with the preferences,
background knowledge and working conditions of
practitioners. Finally most initiatives seem to ‘freeze’ once
the project by which they were initiated stops [11]. During
the user research and evaluations performed during the
development of the UCD toolbox, it became apparent that
very few of the existing method collections are known
among UCD professionals.
An overview is given below of six of these tools; the complete
overview was made available as a card set (see ‘more
information’ at the end of this subchapter).
The method descriptions found in existing method collections
are often incomplete and at times considered ‘too academic’
by practitioners. Most importantly, method collections provide
little or ineffective guidance for selecting the appropriate
method. Some offer a categorization of methods, but these
UsabilityNet
Methods Table UsabilityBOK Usability Planner
Service Design
Tools
Generic Work
Process AllaboutUX
Description Co-created by Nigel
Bevan in 2003 to
‘provide usability
professionals with an
authoritative website
of resources’
Initiated in 2004
by the Usability
Professionals
Association to be a
growing library on
UCD content (not only
methods)
Initiated in 2010
by Nigel Bevan to
support the selection
of UCD methods to be
applied in a project or
organization
Created in 2008
by Roberta Tassi
(graduation student)
further developed by
two Italian universities
in 2009
Created in 2009
by students and
researchers from the
Rotterdam University
of Applied Sciences.
Allaboutux.org
(2010) provides
information about
user experience
(UX) collected and
maintained by
volunteers
Number of
Methods
35 40 - 40 88 82
Selection
mechanism
No No Yes, 7 Filters No No Yes, 6 Filters
Categorization Phase No No No Phase No
Method
description
Text only: Description
varies between
methods. Contains
at least a summary,
benefits, planning,
references next
steps.
Text only: basic
description, (dis)
advantages, how to,
considerations and
references.
Text only: Short
description with (if
available) a link to
the UsabilityBOK
description
Short description with
a reference URL and
case studies
Text only: Short
description and
references to websites
and articles for further
reading.
Text only: Short
description
with strengths,
weaknesses,
references and
characteristics
overview](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-54-320.jpg)
![593.3 UCDtoolbox.com
University webpage of Jasper van Kuijk
More information about his research and educational
activities
http://bit.ly/vankuijk
Contact
If you would like to learn more about the UCD toolbox or
if you would like to contribute by submitting methods or
reviewing them, please visit www.ucdtoolbox.com or
contact one of the authors:
Tristan Weevers
Delft University of Technology
Industrial Design Engineering
UCDtoolbox
t.j.t.weevers@tudelft.nl / tristan@UCDtoolbox.com
Jasper van Kuijk
Delft University of Technology
Industrial Design Engineering
+31 (0)15 278 1378
j.i.vankuijk@tudelft.nl
Core publications
Weevers, T.J.T. (2011) Method Selection Tool for User-
Centered Product Development. Delft University of
Technology. Master thesis.
References
[1] J. I. van Kuijk, “Managing Product Usability; How
companies deal with usability in the development
of electronic consumer products,” PhD thesis, Delft
University of Technology, Delft, 2010.
[2] N. Bevan, “UsabilityNet Methods for User Centred
Design,” in Proceedings of HCI International, Crete,
Greece, 2003, pp. 434-438.
[3] B. Tidball, Stappers, P.J., Mulder, I., “Models, Collections
and Toolkits for Human Computer Interaction: What Can
We Learn?” in Proceedings of the 24th BCS Conference
on Human Computer Interaction, Dundee UK, 2010.
[4] F. Ferre, Bevan, N., Escobar, T.A., “UCD Method Selection
with Usability Planner,” in Proceedings of NordiCHI,
Reykjavik, Iceland, 2010, pp. 829-830.
[5] N. Bevan, “Criteria for selecting methods in user-centred
design,” presented at the I-USED, Upssala, Sweden,
2009.
[6] A. Seffah, Metzker, E., “The obstacles and myths of
usability and software engineering,” in ACM 47 (Dec.
2004), 2004, pp. 71-76.
[7] S. Greenberg, Buxton, B., “Usability Evaluation
Considered Harmful (Some of the Time),” in Proceedings
of CHI 2008 Florence, Italy, 2008, pp. 111-120.
[8] J. Nielsen, “The use and misuse of focus groups.” IEEE
Software [0740-7459], vol. 14, pp. 94-95, 1997.
[9] J. Goodman-Deane, Langdon, P.M., Clarke, S. and
Clarkson, P.J. , “Categorizing design methods: how
designers view the roles of user methods in design,” in
Contemporary Ergonomics 2008 (CE2008), Nottingham,
UK, 2008.
[10] C. Cardoso, Clarkson, P.J., Keates, S., “Can Users
Be Excluded from an Inclusive Design Process?” in
Proceedings of HCI, Las Vegas, 2005.
[11] Battle, L., N. Bevan, et al. “About the Usability Body of
Knowledge.” Retrieved March 26, 2012, 2012, from
http://www.usabilitybok.org/about.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-61-320.jpg)
![64 Design for Usability Methods Tools
Author Bio
Christelle Harkema obtained
her BSc and MSc degree in
Industrial Design Engineering
at the Technical University of
Delft. During her Master’s she
focused on ergonomics and
research. After graduating she worked for 2 years as a
usability specialist for Indes, the Dutch design agency.
Combining practice and theory, she did a PhD on
usability at Eindhoven University of Technology.
Christelle’s PhD was supervised by Assistant Professor
Ilse Luyk, Professor Cees Dorst and Professor Arnout
Brombacher.
some unpleasant surprises during the final tests: users had
difficulties understanding how to operate the product. Design
adjustments and extra user tests were necessary to solve
these issues.
How could this happen? This is a real world example that
took place at a leading multinational with seemingly all the
knowledge and expertise in place to develop successful
products. The multi-disciplinary team consisted of skilled
and experienced members of various disciplines, including
interaction designers and usability specialists. Many usability
principles and techniques were available to them and still
the users experienced usability issues with the product. The
cause, in this case, was that, at the moment of decision-
making, the team was unaware of the consequences of this
decision to use certain buttons to operate the product.
It has been shown that usability issues can be traced back to
decisions made during development projects [2]. However,
there are many other aspects that can influence decision-
making: team dynamics, skills of individual team members,
how knowledge on usability issues is collected, the stage of
the development process; these can all play a role. Above
all these, decision-making can be strongly influenced when
an issue as intangible and fluid as usability is concerned.
This research project was set up to improve usability related
decision-making by identifying the critical factors that
influence the quality of decisions in product development.
Research method
Four studies were executed to explore the factors that
influence the quality of usability-related decision-making.
To gain a better understanding of decision-making and
factors that were known to influence it, we first conducted
a review of the literature. One of the factors identified was
uncertainty: ‘knowing that you do not know’. For example,
you have doubts whether a user test identified all the
Introduction
A team in a room, exploring the possibilities for innovative
technological solutions and creating design ideas for a new
generation of a product. Its predecessor was launched only a
few weeks ago, but its success made the demand for a quick
follow-up necessary. The team is the same as on the first
product and is focused on how to improve the first version.
Unfortunately, what happens next is a series of large
organizational changes within the company, resulting in
serious changes to the composition of the team. This new
team worked hard to implement the conceived technological
improvements and executed usability tests to validate the
improvements made to the user interface. Meeting time
and budget resources were limited, as by now they were
nearing the end of the project. Unfortunately there were](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-66-320.jpg)
![654.1 Preventing unawareness in usability related decision-making
usability issues or you are uncertain whether you picked
the right user interface concept; you are not certain that you
have either all the information and whether it is correct. The
second study was an explorative study at a Dutch design
agency. This study verified that uncertainty is an influencing
factor on decision-making in design practice. However, it
also raised a new question; what if a team is not uncertain
but unaware. What if they are in a state of ‘not knowing that
you do not know’. For example by not even considering that
a particular design could lead to usability issues? In a third
study, involving 14 interviews with core team members of
a product development project at a leading multinational,
it was confirmed that usability issues do result from
unawareness during decision-making in the development
process. The final and fourth study was an in-depth study
to assess how unawareness leads to usability issues. The
starting point for this study were usability issues that were
identified in the product. Studying 2.056 project documents
provided an overview of the process and the usability-related
decisions taken during its course. This resulted in being able
to distinguish between multiple types of unawareness and
the identification of possible sources of unawareness.
Results
Decision-making
Usability issues are a result of a mismatch between how the
user uses a product (actual use) and the how the designer
intended the use with the product (intended use) [1]. Intended
use results from design decisions made during the product
development process. Previous research has shown that
‘incorrect’ design decisions may result in usability issues [2].
Therefore the focus of this project was on usability related
decision-making, (see Figure 1), and on finding ways to
improve this decision-making, thereby reducing the number
of usability issues.
The development of electronic products is often executed
by a team, operating in a certain ‘context’. This context
has influential characteristics, such as time pressure, the
presence of multiple stakeholders, and that the team
has to deal with ‘ill-structured problems’. Time pressure
in product development is generally high, as an early
market introduction will give your company an advantage
in sales and profit [3], while the product development
financial resources are often limited. Secondly, today’s
complex products cannot be designed by a single person,
they require multidisciplinary teams, in which each
stakeholder or discipline has its own viewpoint, knowledge
and values from their own area of expertise [4]. The
designer challenge is to balance and integrate the various
perspectives to come to a good solution. A last example of
characteristics of the context is that design problems are
‘ill-structured problems’, problems that are not fully defined
[5], resulting in many possible solutions, each with its own
advantages and disadvantages.
Figure 1: The focus of this research project: decision-making in design practice](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-67-320.jpg)
![66 Design for Usability Methods Tools
absence of information, certain pieces of information can
be missing, or the information can be unreliable. In addition
to incomplete information, difficulties in understanding can
make decisions harder to take. A good understanding of the
information can be obstructed by ambiguous information,
the novelty of situations, and fast-changing or unstable
situations. Finally it can even be, there are conflicting
alternatives, in which case the solutions have different
advantages and disadvantages. To sum this up, uncertainty
is about the certainty about answers to questions, while
the following factor ‘unawareness’ is about asking the right
questions.
Unawareness
During the exploratory phase of our research, we identified
unawareness as an influencing factor on usability related
decision-making. A subsequent literature study on the
subject of unawareness showed that it is not a common
topic in design literature. Therefore the majority of the studies
Although there is no generally accepted model of decision-
making in design, the following three phases can be
distinguished [6]:
1 Situation recognition: based on team member expertise
the situation is observed and assessed as being typical
or novel, resulting in the choice of either a typical or a
new, more challenging approach;
2 Serial option evaluation: evaluate the options that are
generated and select a satisfactory one;
3 Simulation: imagine whether the selection option and
action will result in a satisfactory solution; this can
be achieved by mental simulation or by sketching or
prototyping.
4 In this process, uncertainty and unawareness are
influencing factors on usability related decision-
making. To improve the quality of decisions, a better
understanding of these factors is necessary.
Uncertainty
The explanation of the uncertainty factor is based on the
results of the literature study and the explorative study in
design practice. Our analysis of uncertainty, we discerned
the following aspects; the types of uncertainty and sources
of uncertainty [7] (Figure 2). The types of uncertainty describe
what the decision maker is uncertain of:
Uncertainty about outcomes
Uncertainty about situation
Uncertainty about alternatives
Sources of uncertainty can be:
Incomplete information
Inadequate understanding
Conflicting alternatives
In our survey of the literature, incomplete information was the
most often mentioned source of uncertainty. Information can
be incomplete in varying degrees. There can be a complete
DECISION-MAKING
sources
incomplete
information
inadequate
understanding
conflicting
alternatives
UNCERTAINTY
uncertainty about outcomes
uncertainty about situation
uncertainty about alternatives
types
contributing
influencing
Figure 2: Types and sources of the influencing uncertainty](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-68-320.jpg)
![714.1 Preventing unawareness in usability related decision-making
References
[1] S. Babbar, R. Behare and E. White, “Mapping Product
Usability.” International Journal of Operations
Production Management vol. 22(10), pp. 1071-1086,
2002.
[2] P.H. den Ouden. “Development of a design analysis
model for consumer complaints: Revealing a new class
of quality failures” PhD Thesis, Eindhoven University of
Technology, 2006.
[3] A.C. Brombacher, P.C. Sander, P.J.M. Sonnemans and
J.L. Rouvroye. “Managing Product Reliability in Business
Processes ‘Under Pressure’” Reliability Engineering and
System Safety, vol. 88, pp. 137-146, 2005.
[4] B. Lawson and K. Dorst “Design expertise.” Oxford:
Elsevier Ltd, 2009.
[5] H.A. Simon. “The structure of ill-structured problems.”
Artificial intelligence, vol. 4(3-4), pp. 181-201, 1973.
[6] Klein, G.A. “A recognition-primed decision (RPD) model
of rapid decision making.” in: Decision making in
action: Models and methods. G.A. Klein, J. Orasanu, R.
Calderwood and C.E. Zsambok, Ed. New Jersey: Ablex
Publishing. 1993, pp.138-147.
[7] R. Lipshitz, and O. Strauss. “Coping with Uncertainty: A
Naturalistic Decision-Making Analysis.” Organizational
Behaviour and Human Decision Processes 69(2), pp.
149-163, 1993.
Contact
If you would like to know more about this research project
or would like to hold an ‘unwareness workshop’ at your
organization, you are welcome to contact the author:
Christelle Harkema
Eindhoven University of Technology
Industrial Design
christelleharkema@gmail.com
Core publications
Harkema, C.L.E. (2012). Revealing unawareness in
usability related decision-making. Eindhoven: Technische
Universiteit eindhoven. PhD Thesis.
Harkema, C.L.E., Luyk-de Visser, I.M., Dorst, K. and
Brombacher, A.C. (2011) ‘Can existing usability techniques
prevent tomorrow’s usability problems? In: Proceedings
of International Conference on Engineering Design,
Copenhagen.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-73-320.jpg)
![754.2 Envisioning Use
For example, when designing a digital camera, it is important
to consider, but hard to predict, the users’ expertise and
preferred level of menu complexity. It is also important to
consider the influence of changing circumstances of use,
such as weather conditions. These affect whether a screen
is readable in bright sunlight, whether the buttons are
controllable with gloves on when it is freezing, or whether the
camera feels slippery with sweaty hands from the heat.
Clearly, knowledge about the context of product use and
about users is essential input into the product development
process. Such knowledge could help when making design
decisions that improve the usability of the resulting products,
particularly when the knowledge is shared between team
members. This study and the resulting workshop aim to
support this sharing and application of knowledge.
To gain insight into current product development practice
and how it handles usability and knowledge about usability,
we studied the product development process of design
teams. The second goal of our research was to come up
with improvements for the problems the design teams
encountered. Based on the insights gained, the Envisioning
Use workshop technique was developed, evaluated and
refined. The technique serves to improve usability during
the development process by providing teams with a shared
vision on product use.
Research method
In order to gain insights into product development practice,
and how designers deal with usability, we conducted
fifteen in-depth interviews with industrial designers [1, 2],
and developed three case studies for product development
practice [3].
The interviews were conducted at the designers’ place
of work, and during the conversation the designers used
previously designed products as tangible examples.
In the case studies, three design projects were studied
retrospectively by means of a group interview and individual
interviews with members of the design team [3]. Related
research by Harkema [4] was the source of case studies on
product development practice. The results of our research
are summarized below.
To overcome the limitations identified through the interviews
and case studies, we developed the Envisoning Use
workshop. The technique was developed iteratively over a
period of two years, incorporating six iterations and three
evaluations. Half-day workshops were held with companies
participating in the DfU project, as well as with other
companies. The evaluations were conducted using fictional
design cases (a presentation microphone and a digital
camera) and real design cases provided by the participating
companies, ranging from business software to household
appliances (Figure 1).
Figure 1: The Envisioning Use workshop was developed through an iterative
approach, alternating evaluations and adjustments of the technique.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-77-320.jpg)
![76 Design for Usability Methods Tools
or not appropriate for this thing.” - Interaction
designer
While designers do share use-related knowledge
amongst each other, there is not enough sharing of their
implicit knowledge with other product development team
members:
“The designer mainly [analysed the user and
the use context]. He did market research
and made critical scenarios, but we didn’t
make these very explicit in this project I think.
I checked again, but really it was just in our
heads.” - Product development team member
Results (2): the Envisioning Use workshop
To facilitate the exchange of implicit knowledge about product
use and users, we developed the Envisioning Use workshop.
Results (1): Interview and Case Study Research
Our interview and case study research led us to conclude
that product development teams do not have enough
direct access to the context of use. They rely on their implicit
knowledge and personal experience. They hardly ever
share this knowledge because there are few opportunities
for successful communication of this ‘informal’ type of
knowledge. Implicit knowledge is difficult to address in
formal design review meetings and procedures. A second
consequence of the knowledge being implicit is that teams
do not clearly distinguish between what they know and
what they assume about product use. More knowledge
sharing in product development teams would lead to better
consideration of usability in design decisions, as has been
concluded elsewhere [4, 5]. This sharing should happen
informally and through a diversity of approaches that
reveal different aspects of knowledge about use.
We found that:
Designers involved in product development often have
little or no direct contact with the actual use situation
and the end-users. Formal usability testing by experts,
though well established, often confirms already known
problems - too late for correction or too low in priority:
“So when we tested the prototypes, we
already knew that they were too heavy.” -
Design manager of consumer electronics
Designers often rely on their implicit knowledge and
experience of usability when making decisions. They
feel responsible and conduct informal user testing
themselves:
“And then I’d ask my girlfriend at home: ‘Do
you like it or not?’ But in general I think it’s
part of my job to decide what’s appropriate
Figure 2: Through the steps of the Envisioning Use workshop, implicit
knowledge of team members is made explicit in a product mind map](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-78-320.jpg)
![774.2 Envisioning Use
Approach and principles
This half-day workshop supports the creation of a shared
vision on product use by:
Getting the members of a product development team
to examine their design task from the user’s viewpoint
rather than the developer’s viewpoint;
Creating an informal environment in which team
members can express and share their use-related
knowledge and experience in several interactive ways.
Use of this technique will not replace user testing; it can
be seen as an ‘add-on’. Existing insights form useful input
for the workshop, and the workshop can itself reveal a
need for further user research.
The basic principles of the workshop are:
Eliciting real-life stories and envisaged scenarios [6, 7];
Making both facts and assumptions explicit: what
participants know and don’t know about product use [4];
Figure 3: Selection of use phases on flip chart sheets Figure 4: Remembering experiences of use
Structuring the information in a ‘product use mind
map’.
Workshop steps
In the Envisioning Use workshop, team members access
their personal knowledge and assumptions about product
use in seven steps (Figure 2):
1 Remembering
2 Imagining
3 Structuring
4 Experiencing
5 Targeting
6 Envisioning
7 Questioning
At the end of the workshop the available implicit
knowledge about product use is made explicit in a shared
frame of reference; the product use mind map, which can
then be used to decide which actions are required during](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-79-320.jpg)
![Figure 11: an example of a wrap up step: assigning actions to be
taken.
814.2 Envisioning Use
“You get an amazing amount of output, which
is really helpful.” – Participant at design studio 1
Doing the workshop together keeps project team
members dedicated to usability in the design process.
“By experiencing this workshop, everyone
who is working on the project stays dedicated
and motivated.” – Participant at design studio 2
Knowledge can be shared in the team on topics that are
familiar to experts, but unfamiliar to the others.
“I am not part of the project, so I don’t know
anything about [certain issue of the target
market], but within four hours I do have an
idea of what is playing a role there.”
– Participant at software company
Validation in practice
Following the six pilots with fictive cases, three half-day
Envisioning Use workshops were held with real cases
in companies, two on household appliances and one
on business software (Figure 1). After each edition the
workshop was evaluated and adjusted accordingly. The
workshops were recorded on video and the participants’
actions analysed from the video. Directly, after each
workshop, the participants reflected on it in a group
interview; later they filled in a questionnaire to assess the
workshop’s effects.
The workshop technique was found to be applicable to
both projects aimed at redesigning an existing product
and projects that have a more explorative character.
With small adjustments it was found to be effective in
all the cases in which it was applied. With products like
household appliances, we conducted the workshop both
Benefits
In our initial research project, we identified a need for more
sharing of use-related knowledge in product development
processes. Individuals make small or large decisions based
on knowledge and experience that remains implicit if it
does not have a clear place or is not easy to back up. The
simple and quick Envisioning Use technique enables teams
to share knowledge, reflect on decisions and to consider
usability earlier on and more easily in the development
process. To be most effective, the first workshop should be
conducted at or before the start of a design project.
The workshop efficiently creates an overview of the use-
related knowledge and knowledge gaps in the design
of a product or service.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-83-320.jpg)
![834.2 Envisioning Use
Contact
The researchers are available to run a workshop as an
introduction to the technique. It can be customized to fit any
design topic.
Stella Boess
Delft University of Technology
Industrial Design Engineering
+31 (0)15 278 3196 s.u.boess@tudelft.nl
Mieke van der Bijl-Brouwer
University of Twente
Engineering Technology
+31 (0)15 27 82551
m.vanderbijl-brouwer@ctw.utwente.nl
Christelle Harkema
Eindhoven University of Technology
Industrial Design
christelleharkema@gmail.com
Core publications
Van der Bijl-Brouwer, M., S. Boess and C. Harkema
(2011). What do we know about product use? A technique
to share use-related knowledge in design teams.
Proceedings of IASDR 2011, the 4th World Conference on
Design Research. Delft (NL).
Van der Bijl-Brouwer, M. and S. Boess (2010). From
remembering to envisioning product use: an informal
design technique. Proceedings of the 7th International
Design Emotion Conference 2010. Chicago (IL, USA).
References
[1] S. Boess. “Experiencing product use in product
design.” in Proceedings of International conference
on engineering design, ICED’09. Stanford University,
Stanford, CA, USA, 2009.
[2] S.U. Boess. “Meaning in product use: which terms do
designers use in their work?” in Proceedings of DeSForM,
Offenbach, Nov 6-7 2008, pp. 20-27, 2008.
[3] M. van der Bijl - Brouwer and M. C. van der Voort.
“Strategies to design for dynamic usability.” Proceedings
of IASDR2009 Design Rigor Relevance. Seoul, Korea,
Korea Society of Design Science, 2009.
[4] C. Harkema, Subchapter 4.1 in this volume.
[5] J.I. van Kuijk. “Managing Product Usability - How
companies deal with usability in the development
of electronic consumer products.” PhD thesis, Delft
University of Technology, 2010.
[6] T. Erickson. “Five Lenses: Toward a Toolkit for Interaction
Design.” In: Theories and Practice in Interaction Design.
S. Bagnara, G. Crampton Smith and G. Salvendy, Ed.
Lawrence Erlbaum Associates, 2006, pp.301-310.
[7] J.M. Carroll. Making use: scenario-based design of
human-computer interactions. London: MIT Press, 2000.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-85-320.jpg)
![86 Design for Usability Methods Tools
Introduction
From the time that consumer electronics were launched
on the market, consumers have complained about many
of these products. Initially, these complaints focused on
technical failure or product-malfunction, however over the
years the numbers of these types of issues slowly decreased,
as did the number of complaints. However, the late 90s
witnessed an increase in consumer complaints, but this
time the cause of the complaints did not seem to lie in
technological failures [1]. According to recent studies, about
half of the products that are returned are not actually broken.
The products are suffering from what is known as ‘soft
reliability problems’, a term introduced by Brombacher et Al.
[2] to refer to the problems that cannot be traced back to a
specification violation. So, the product is fully functional, does
what it is intended to do, but yet users return it.
Consumer dissatisfaction with regard to soft problems is
presumably a consequence of a mismatch between user
and designer/company expectations in product use [3].
Based on brand identity, use context, product properties and
mediated by user characteristics (e.g., personality traits),
users form expectations regarding the usability of a product
[4]. However, the initial expectations that consumers have
might differ from what they experience when actually using
the product. Negative disconfirmation (underperforming in
relation to expectations) leads to feelings of dissatisfaction
(Figure 1).
One of the reasons why the causes for the increase in
complaints and product returns due to soft reliability are
unknown is that it is common practice in the consumer
electronics industry that customer complaints are dealt with
by call centres. There are often very few direct links between
these centres and the product development departments [5].
Figure 1: Conceptual framework of the study explaining how product and user
characteristics interact and result in expectations about product use, which are
then (dis)confirmed by actual product use.
Author Bio
Cha Joong Kim is working
on his PhD at the Faculty of
Industrial Design Engineering,
Delft University of Technology
in the Netherlands. He holds
an MSc in Industrial Design
Engineering from the same university. Since 2007 he
has been involved in the Design for Usability project.
His main research interests include user diversity,
cultural differences in design, and cognitive aspects
of human-product interaction.
Cha Joong Kim’s PhD project was supervised by
Associate Professor Henri Christiaans and Professor
Daan van Eijk.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-88-320.jpg)
![874.3 Anticipating soft problems
Despite this increase in soft problems, only a few studies
have investigated which soft problems consumers encounter,
let alone explain how user characteristics, product type and
soft problems interact with each other. This project focused
on improving the theoretical foundation and finding empirical
evidence for soft problems.
Research method
Based on our collected knowledge, a design method
was devised to help product development teams better
understand the interaction between user, product and soft
problems.
First, we developed a conceptual framework based on the
primary variables and the relations between them believed
to play a role in the experience of soft problems (Figure 1).
Next, for the evaluation of this framework, we conducted
a total of four practical studies (three questionnaire-based
surveys and a laboratory experiment). The first study
identified which types of soft problems people experience.
The second study explored mainly how user characteristics
are related to specific types of soft problems. In order to
study product type in relation to user characteristics more
directly, we conducted a laboratory experiment. In this
experiment participants were asked to interact with two
products, an alarm clock and MP3-player, which were known
to suffer from a low level of usability. The last study, again a
questionnaire-based survey, was used to validate findings
from the preceding studies and experiment.
Results
Types of soft problems faced by consumers.
The first study mapped non-technical problems for a
technically sound product which users experienced when
interacting with household electronic products and services,
and their reasons for complaining (or not). Interestingly, all
the complaints are related to instrumental interaction, i.e.
using, operating, and managing products [6]. Using product
quality theory [7, 8], the types of problems found were
categorized into three groups on the basis of the reasons
why people were frustrated by their product(s) (Figure 2).
Sensory quality
This interaction quality is related to sensory perception.
Consumers use their perceptive faculties to assess the
structure, visibility, weight, sound, texture, and smell of a
product. Judging this quality happens instantaneously while
using, it is momentary, and based on human senses and
can lead to either a pleasant or unpleasant experience.
User dissatisfaction related to sensory quality is related to
awkward product structure, visual hindrance, over- or low-
weight, noise, irritating touch, and unpleasant smell.
Functional quality
This interaction quality refers to the extent to which users
achieve their goals with the product. It is evaluated by
assessing the results achieved after prolonged product use.
Accordingly, the appreciation of this quality is not immediate
but it’s effect is lasting. Complaints related to interaction
quality mostly result from technological limitations or from a
lack of product durability: for instance, functional constraints
such as lack of function and incompatibility, low performance
such as slow reaction and short battery life, unexpected
errors, and frequent breakdowns. They are also related to
poor product service.
Operational quality
Operational quality is related to cognitive and physical
efforts that users have to invest to operate or interact
with a product. The assessment of this interaction quality
happens immediately and is long-lasting. Users evaluate
ease of use, and the need for maintenance and repairs.
Complaints related to operational quality surface if using
the product requires continuous cognitive efforts or if users](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-89-320.jpg)
![954.3 Anticipating soft problems
References
[1] P.H. Den Ouden, “Development of a design analysis
model for consumer complaints: revealing a new class
of quality failures,” PhD thesis, Eindhoven University of
Technology, Eindhoven, 2006.
[2] A. C. Brombacher, P. C. Sander, P. J. M. Sonnemans, and
J. L. Rouvroye, “Managing product reliability in business
processes ‘under pressure’,” Reliability Engineering and
System Safety, vol. 88, pp. 137–146, 2005.
[3] E. den Ouden, L. Yuan, P. J. M. Sonnemans, and A. C.
Brombacher, “Quality and reliability problems from
a consumer’s perspective: an increasing problem
overlooked by businesses?,” Quality and Reliability
Engineering International, vol. 22, pp. 821-838, Nov
2006.
[4] S. Donoghue and H. M. de Klerk, “Dissatisfied
consumers’ complaint behavior concerning product
failure of major electrical household appliances - a
conceptual framework,” Journal of Family Ecology and
Consumer Science, vol. 34, pp. 41-55, 2006.
[5] A. Koca, “Soft Reliability in New Product Development:
An Ontological Approach for Utilizing Field Feedback
to Dynamically Sense and Adapt to Evolving Global
Markets,” PhD Thesis, Eindhoven University of
Technology Eindhoven, 2009.
[6] P. Hekkert and P. Desmet, “Framework of Product
Experience,” International Journal of Design, vol. 1, pp.
57-66, 2007.
[7] D. Dantas, “The role of design to transform product
attributes on perceived quality,” in Proceedings of the 6th
International Congress on Design Research, Lisbon, 2011.
[8] O. M. Madureira, “Product standardization and industrial
competitiveness,” in Proceedings of the International
Conference on Standardization and Quality, Sao Paulo,
1991, pp. 119-124.
More information
Card set
Provides an overview of which user characteristics and
product properties lead to which soft problems
http://bit.ly/up-cards
Contact
If you are interested in learning more about this research,
method or workshop, please contact the author.
Cha Joong Kim
Delft University of Technology
Industrial Design Engineering
C.Kim@tudelft.nl
+31 (0)15 27 85146
Core publications
Kim, C. Christiaans, H. (in press). User characteristics
and behaviours in operating annoying electronic
products. International Journal of Design.
Kim, C., Christiaans, H. (2012). Soft problems with
consumer electronics: the interaction between user
characteristics and usability. Journal of Design Research,
10(3), 223-238.
Kim, C., Christiaans, H. (2011). Usability problems:
the influence of user diversity. IASDR 2011, Delft, the
Netherlands. (selected as reviewers’ favourite)](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-97-320.jpg)
![99
5.1
Guidelines to design for dynamic and
diverse use situations
Exploring the who, where and why of
product use
Mieke van der Bijl-Brouwer
American tourist Brian Wilson stood on a street corner in Amsterdam for
73 minutes and captured what to him was an amazing diversity of bicycle
riders and use [1]
Summary
The aim of this study is to support product development
teams in dealing with the variety of situations in which
products are used, so-called dynamic and diverse use
situations. Dealing with varying use situations in the design
process is difficult because it is hard to predict the situations
in which a product will be used, to anticipate what will
happen when the product encounters those situations
and to generate solutions for conflicting requirements.
Our retrospective study of three design projects in practice
showed that knowledge of dynamic and diverse use
situations often remains implicit and is not shared between
members of a product development team. This can have a
negative effect on the validity of usability evaluations and
can give rise to difficulties in decision-making with regard to
product usability. We therefore developed a set of guidelines
to support teams when dealing with dynamic use in the
design process. The guidelines were developed iteratively
and evaluated in seven student projects. They enable teams
to create an explicit frame of reference of use situations
which can be applied to contextualize usability evaluations;
a ‘dynamic use mindset’ which inspires solution generation;
and a shared vision on product use which supports decision
making.
YouTube
Watch a 2½ minute
animated summary
of this research:
http://bit.ly/ddus-summary Research Findings](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-101-320.jpg)
![100 Design for Usability Methods Tools
Design for DDUS is difficult because it is hard to:
predict the variety of use situations a product will
encounter: use situation analysis;
anticipate what kind of issues will occur when the
designed product interacts with these situations: use
anticipation;
deal with conflicting requirements from the different use
situations in one design: solution generation.
The relation between the different aspects of design for
DDUS – use situation analysis, use anticipation and solution
generation - is illustrated by the following example (See
Figure 1). Designers of smart phones will undoubtedly do
a great deal of research on who uses or would like to
use smart phones, for which purposes, and under which
circumstances. Apple designers might have expected that
people would use the iPhone outside, also in cold weather
(use situation analysis). However, that Korean people would
use sausages to operate the phone in cold weather (use
anticipation) was probably unforeseen! As the example
demonstrates, each specific use situation can require
different product characteristics. In this case, the problem
can be solved by providing an accessory for the iPhone, such
as a special glove (solution generation).
Another difficulty of design for DDUS is that decisions need
to be made on which use situations will be taken into
account and which solution proposals will be chosen to suit
these use situations. To make these decisions in a product
development team, team members should have a shared
understanding of use situations and related use issues.
The importance of considering the variety of use situations
is recognised in the main literature on usability. Many
techniques are available to analyse use situations, such as
interviews and observations [2], probing [3, 4] and after sales
feedback [5]. Furthermore, the literature often mentions that
the test conditions of usability evaluations should represent
Introduction
As opposed to tailored products, industrially manufactured
products are used by varying users, for varying purposes
in varying contexts. I have termed this dynamic and diverse
use situations (DDUS). These situations refer to the change
of situations over time for one product, for example, one day
you might use your bicycle to quickly cycle to university to get
to a lecture on time, while the next day you might use it to
transport your groceries from the supermarket to your home.
DDUS refer to the change of situations in time and space for
different versions of the same product. For example, someone
else might possess the same type of bike but only use it for
recreational purposes, for example cycling with friends.
Author bio
Mieke van der Bijl-Brouwer
graduated cum laude in Industrial
Design Engineering at TU Delft
in 2002. She is now an Assistant
Professor at the University of
Twente and researches and
lectures on subjects in the field
of usability, user experience and
scenario-based design. She was awarded her PhD for
her research on ‘Design for Dynamic and Diverse Use
Situations’ (cum laude) in September 2012. She also
contributed to the development of the Envisioning Use
technique (see subchapter 4.2)
Mieke’s PhD was supervised by Associate Professor
Mascha van der Voort and Professor Fred van Houten.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-102-320.jpg)
![101
the actual user, goal and environment [6, 7]. However,
in spite of these acknowledgements, little guidance
is available on how an analysis and specification of
intended use situations can lead to a frame of reference
for usability evaluations. The purpose of our study
therefore, was to develop a support tool aimed at filling
this gap.
Research method
To analyse how designers in practice currently deal
with DDUS, a retrospective study of three real-world
design projects was conducted. Information about the
projects was gathered by means of group and individual
interviews with members of the development team of a
product with varying use situations.
Our analysis led to the problem definition, which we used as
input for the development of a support tool which enables
designers to deal with knowledge of DDUS in the design
process. The tool consists of the Envisioning Use technique,
which is discussed in Chapter 3.2, and a set of guidelines.
The guidelines were developed iteratively in two educational
projects, in which students designed for a real client. In the
first project, four student teams designed a carrier bike for
Bongo Innovations BV. The design processes of both projects
were analysed by means of document analysis and a group
interview. The guidelines were then revised based on these
insights. In the second project, three student teams, with
the help of the revised guidelines, redesigned an Airfryer for
Philips. An evaluation of both projects led to a final workbook
with guidelines.
5.1 Guidelines to design for dynamic and diverse use situations
GOAL
USER
GOAL
USER
CONTEXT
solution generation: creating solutions for conflicting requirements
usability evaluation: evaluating and anticipating interactions and related usability issues
use situation analysis: predicting and prioritising situations in which products will be used: users, goals and contexts
use issues use issues
use issue use issues
use issues
use situation: korean
person in cold
wearing gloves
wants to use iPhone
product: iPhone
solution to
accomodate
this situation:
special glove
issue: people cannot
control touch screen
with gloves, instead
they use a sausage
Example:
PRODUCT
interaction
interaction interaction
interaction
GOAL
USER
CONTEXT
GOAL
USER
CONTEXT
Figure 1: explanation of the different theoretical problems when designing for diverse use situations.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-103-320.jpg)
![105
fit diverse use situations by stimulating the integration of use
situations in the usability evaluations, which in turn can lead
to use situation-specific recommendations for the creation of
solutions.
The activities mentioned in the previous sections are not
meant to replace current design activities. On the contrary,
they refer to activities that often implicitly – already occur in
practice. The added value for design for DDUS is that the
relation of these activities to the frame of reference with
DDUS now becomes more apparent. The guidelines can
therefore be applied in existing design approaches.
Validation
The guidelines have been evaluated in educational projects,
as described in the research method. A validation in student
projects enables close observation of the design process
by multiple respondents. Their evaluations have led to the
conclusion that working with an explicit frame of reference
supports the generation of more focused research questions
in usability evaluations, and offers opportunities for setting
up more valid test conditions. A valuable approach to
generating the explicit frame of reference is combining
the exploration of the relation between use situations and
usability with the verification of this relation. Moreover, the
joint creation of this explicit frame of reference (such as
within the Envisioning Use workshop) has led to a shared
vision on product use in the design teams. Students highly
valued this shared vision on product use, and found it
beneficial in their team decision-making processes and in
their creative process of solution generation. The workbook
format of the guidelines did not always have the intended
effects. Therefore future research will be aimed at developing
other formats to improve the usability of the guidelines.
Limitations
Applying these guidelines will not lead to a company starting
activity can be used to add, verify or remove information on
product use, the frame of reference evolves in the course of
the design process.
Applying the frame of reference in usability evaluations
The main advantage of a complete explicit frame of
reference is its application in usability evaluations. Targeted
use issues in the frame of reference can be translated into
research questions for usability evaluations. For example, a
question for the design of the hood for the carrier bike could
be: ‘Can children communicate sufficiently with parents
while seated in the box?’
The most important function of the frame of reference when
planning usability evaluations is that it helps to set proper
test conditions. To increase the ‘external validity’ of usability
evaluations, the test conditions of these evaluations should
reflect actual use situations as much as possible [8, page
241]. The frame of reference should give insights into what
these actual use situations are. For example in the case
of the carrier bike, apart from testing in different weather
conditions, it makes sense to invite children of varying ages
for a user test because their needs for communication with
parents while seated in the box can be assumed to differ.
Benefits
The ultimate goal of the guidelines is to develop products
with a high level of usability in the targeted diverse- use
situations. This can be achieved by, on the one hand,
creating design proposals with a high level of usability and,
on the other hand, by better decision-making processes
with regard to choosing the most appropriate solution and
target use situations. The latter is achieved by stimulating
the creation of a shared vision on product use. The former
is achieved by firstly stimulating the designer’s awareness
of dynamic and diverse use situations and thereby creating
a mindset which can inspire solution generation. Secondly
the guidelines support the creation of solutions that better
5.1 Guidelines to design for dynamic and diverse use situations](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-107-320.jpg)
![108 Design for Usability Methods Tools
Core publications
Van der Bijl-Brouwer, Mieke: Exploring Usability, design
for dynamic and diverse use situations, University of
Twente, PhD thesis, 2012
Van der Bijl-Brouwer, Mieke: Exploring design for
dynamic use, proceedings of the international conference
on engineering and product design education, 8 9
September 2011, City University, London, UK , 2011
Van der Bijl-Brouwer, Mieke and van der Voort,
Mascha C.: Strategies to design for dynamic usability in
Proceedings of IASDR2009 Design Rigor Relevance,
Oct. 18-22, Seoul, Korea, 2009
References
1] B. Wilson. (2006, 10 September). Amsterdam Bicycles.
Available: http://www.ski-epic.com/amsterdam_
bicycles/index.html
[2] H. Sharp, Y. Rogers, and J. Preece, Interaction Design:
Beyond Human-Computer Interaction, 2nd Edition ed.
John Wiley Sons, Ltd, 2007.
[3] B. Gaver, T. Dunne, and E. Pacenti, “Design: Cultural
probes,” Interactions, vol. 6, pp. 21-29, 1999.
[4] S. A. G. Wensveen, “Probing experiences,” in
Proceedings of the first international conference on
Design Emotion, Delft University of Technology, 1999.
[5] J. I. van Kuijk, H. Kanis, H. H. C. M. Christiaans, and D.
J. van Eijk, “Usability in Product Development Practice:
After Sales Information as Feedback,” in Proceedings of
IASDR07, Hong Kong Polytechnic University, 2007.
[6] J. Rubin, Handbook of usability testing. New York: John
Wiley Sons, 1994.
[7] N. Bevan and M. Macleod, “Usability measurement in
context,” Behaviour and information technology, vol. 13,
pp. 132-145, 1994.
[8] P. W. Jordan, B. Thomas, B. A. Weerdmeester, and I. L.
McClelland, Eds., Usability evaluation in industry. Taylor
Frances, 1996, p.^pp. Pages.
PhD Thesis
Exploring usability, design for dynamic and diverse use
situations
http://bit.ly/ddus-thesis
Author homepage
Homepage of Mieke van der Bijl-Brouwer, highlighting
education and research activities
http://bit.ly/vanderbijl
Contact
If you are interested in applying the guidelines in design
practice, please contact the author for inquiries and support.
Mieke van der Bijl-Brouwer
University of Twente
Engineering Technology
+31 (0)15 27 82551
m.vanderbijl-brouwer@ctw.utwente.nl](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-110-320.jpg)
![112 Design for Usability Methods Tools
Introduction
In order to design useful and user-friendly products, it is
important to understand user needs and characteristics.
However, it is equally important to see how technologies
guide and change users. Clever use of these effects
can improve usability and product acceptation whereas
neglecting them often leads to product failure [1, 2].
A good case is the new electronic payment system
introduced by Dutch public transport carriers (OV chip
card). The system has been introduced nation-wide in the
Netherlands by all the public transport carriers in all busses,
Author bio
Steven Dorrestijn followed a
two-year program in Mechanical
Engineering and Design History.
He holds a Master’s in Philosophy
of Science, Technology and
Society from the University
of Twente, and an additional
Master’s in Philosophy from the
University of Paris, Nanterre. His interest in product
design and social aspects of technology were
combined in his research on Product Impact, for
which he was awarded a PhD from Twente University
in October 2012.
Steven’s PhD was supervised by Professor Peter-Paul
Verbeek and Professor Hans Achterhuis.
Figure 1: If the extraordinary curve in this bicycle lane in Paris makes you
smile, this may also suddenly make you aware of to what a great extent our
everyday movements are guided and constrained by technology.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-114-320.jpg)
![1135.2 The Product Impact Tool
trams, the subway, and trains.. There have been many issues
since its introduction some of which have been made news
headlines, for example privacy issues concerning travellers’
data and hacking of the RFID chips by university researchers.
When the public at large was introduced to the system in
2009, practical user problems also attracted a great deal of
critical attention, especially the problem of forgetting to check
out, a new and extra procedure compared to the old paper
ticket system.
The problem of forgetting to check out can be illustrated by
my own experiences. Being curious about the OV chip card, I
was happy to try it as soon as the system was first introduced
in Rotterdam and Amsterdam in 2009. At first, the standard
procedure for charging the card, and getting on and off a bus
or tram seemed self-evident and easy. However, all the rest
seemed to be quite difficult: extra subscription procedures for
first use on the trains, very unclear installation of automatic
money recharge, etcetera. Later on, I also found that the
basic procedures for checking in and out caused major
problems. After I had used the card a few times, I no longer
felt confident about its use and was far from sure that I was
using the card correctly. When I got a printout of my travel log
at a machine, it appeared that I had made mistakes checking
out and changing trams on all the four occasions that I
had used the card. Every time people check in, a deposit is
taken from the card. I had lost the 4 euros deposit on four
occasions. My clumsiness was no exception. In September
2010 it appeared that the public transport companies had
received half a billion Euros in deposit money as a result of
these ‘incomplete transactions’ [5].
The case of the OV chip card is a clear example of
a mismatch between the technical possibilities and
characteristics and the consumer needs and practices. Partly,
this mismatch stems from technical setbacks that can be
overcome. However the OV chip card case also demonstrates
the implications of technology on the way of travelling,
and how the efforts required from consumers to learn new
routines were completely underestimated.
Technology should not simply be considered as a neutral
means to fulfil needs that were always already there. Instead,
technology can change users: it affects their behaviour, as
well as their attitudes, needs, their vision on life, and way
of living. The user-guiding and user-changing effects of
technology form an important research topic in reflexive
research fields – from philosophy and history to psychology
[6, 7, 8]. To date, design practice has made little use of this
knowledge, but there is a growing awareness of the possible
advantages of combining these research fields [9, 10, 11]. This
combination of both perspectives is innovative and promising
for enhancing human-technology interaction and usability.
This chapter presents the Product Impact Tool which was
developed as part of the Product Impact research project.
The OV chip card (Figure 2) case serves to illustrate how
an analysis of behaviour guiding and changing effects of
technology can contribute to increasing understanding and to
reducing problems of usability and technology acceptation.
Figure 2: Extra signage placed
during the introduction of the OV
chip card system.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-115-320.jpg)
![1195.2 The Product Impact Tool
References
[1] D.A. Norman. The psychology of everyday things. New
York: Basic Books, 1988.
[2] P.-P. Verbeek and A. Slob. User behavior and technology
development: Shaping sustainable relations between
consumers and technologies. Dordrecht: Springer, 2006.
[3] P. van den Broek. “De schokgolf na de ontmanteling.”
Vox 8(15), pp. 14-18, 2008.
[4] J. Koot, “Reizigers verliezen iedere maand half miljoen
euro door chipkaart,” Financieel Dagblad, 24 September,
2010.
[5] P.-P. Verbeek, What things do: Philosophical reflections
on technology, agency, and design. Pennsyvania:
Pennsyvania State University Press, 2005.
[6] M. McLuhan. Understanding media: The extensions of
man (Critical edition by W. T Gordon). Corte Madera, CA:
Gingko Press, 2003
[7] E. Tenner. Our own devices: The past and future of body
technology. New York: Alfred A. Knopf, 2003.
[8] N. Tromp, P. Hekkert P-P. Verbeek, “Design for Socially
Responsible Behavior: A Classification of Influence Based
on Intended User Experience.” Design Issues 27(3), pp.
3-19, 2011.
[9] R.H. Thaler C.R. Sunstein. Nudge: Improving decisions
about health, wealth, and happiness. New Haven: Yale
University Press, 2008.
[10] B.J. Fogg, B. J. Persuasive technology: Using computers
to change what we think and do. Amsterdam; Boston:
Morgan Kaufmann Publishers, 2003.
Author homepage
More information on the author’s research and educational
activities:
http://bit.ly/sdorrestijn
Contact
If you are interested in learning more about the Product
Impact Tool or would like to hold a Product Impact session at
your organization, please contact the author.
Steven Dorrestijn
Twente University
Department of Philosophy
+31 (0)53 4894811
s.dorrestijn@utwente.nl
Core publications
Dorrestijn S. (2012). The design of our own lives: Technical
mediation and subjectivation after Foucault. PhD Thesis.
Twente University, Enschede, The Netherlands.
Dorrestijn, S. (2012). ‘Technical Mediation and
Subjectivation: Tracing and Extending Foucault’s
Philosophy of Technology’. In: Philosophy Technology,
25 (2), pp. 221-241.
Dorrestijn, S. (2011) ‘Gedragsbeïnvloedende techniek en
usability’. In: Tijdschrift voor ergonomie36/1, pp. 5-12.](https://image.slidesharecdn.com/4b439223-e7c5-4b1a-93e8-4e80788e7fd2-150624075612-lva1-app6892/85/DfUbook_methods-tools-121-320.jpg)
DfUbook_methods-tools
- 1. Design for Usability Methods & Tools A practitioner’s guide
- 2. IOP IPCR Design for Usability research project Delft University of Technology Faculty of Industrial Design Engineering TU/Eindhoven Department of Industrial Design University of Twente Faculty of Engineering Technology and Department of Philosophy The authors gratefully acknowledge the support of the Innovation- Oriented Research Programme ‘Integral ProductCreation and Realization (IOP IPCR)’ of the Netherlands Ministry of Economic Affairs, Agriculture and Innovation. Editing Jasper van Kuijk, Roger Staats Design and layout Marieke de Roo Grafisch Ontwerp, Delft Photography Cha Joong Kim, Job Jansweijer, Tjebbe van Eemeren, Jasper van Kuijk Printer DeltaHage, Den Haag © 2012 Design for Usability All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic, or mechanical, including photocopy, recording or any other information storage and retrieval system, without prior permission from the publisher. ISBN 978-94-6186-077-4
- 3. Contributions by Stella Boess Mieke van der Bijl-Brouwer Steven Dorrestijn Daan van Eijk Christelle Harkema Frederik Hoolhorst Cha Joong Kim Jasper van Kuijk Mascha van der Voort Tristan Weevers Edited by Jasper van Kuijk Design for Usability Methods & Tools A practitioner’s guide
- 4. 2 Design for Usability Methods & Tools
- 5. 3Introduction Prof Daan van Eijk Over the past five years, it has been our pleasure to dive deep into the practice of usability and user-centred design. The Design for Usability IOP-IPCR research project was set up to improve the ability of companies to create usable products. We believe we’ve made a valuable contribution to resolving many issues and questions faced by user-centred design practitioners in their efforts to design more user-friendly products. Happily, this feeling has been substantiated by the reactions we’ve received from practitioners. You are our users. The purpose of the new knowledge and methods developed through the Design for Usability project is that they are to be applied by you, during the design and development of electronic consumer products. We believe that research into design and product development is of great value only if the outcomes find their way back to practice. This is why we involved four companies as partners in the project, and why we interviewed so many practitioners, performed case studies, and evaluated our new methods and tools with a community of practitioners. It is also why we invested time and effort in making sure the results of our studies found their way into practice, for example through an Dear Practitioners up-to-date, accessible project website filled to the brim with content; through the yearly Design for Usability symposia on World Usability Day; and by communicating research outcomes innovatively and enjoyably using, for example, animated videos. And now there is this book, Design for Usability: Methods & Tools, in which you will find a comprehensive and coherent overview of the DfU project results. This book is not intended to be a scientific publication or a ‘usability handbook’; it is a means for sharing the methods and knowledge generated within the DfU project. It also serves as a physical link to the content available on the website. We have written it for a readership of product design and development professionals who want to improve the user- centred product development capability of their organization. Targeted roles include: > Usability/user experience specialists Interaction/product designers Product/project managers Upper management (business group management, design managers)
- 6. 4 Design for Usability Methods Tools The focus of this book is on the practical project results, the methods and tools, which are presented in a way so that you can assess how they can be of value to you and your colleagues. And if you’re interested in applying them, each chapter includes links which lead you to more information and guidance or to the people behind the projects, who will be happy to tell or show you more. Of course we also want to describe the process of how we developed our new methods and tools. The Design for Usability project builds on existing knowledge in the domains of usability, user experience, and user-centred design. Our aim is to make our results applicable in practice, and we demonstrate that our solutions are based on the valuable work that design researchers have done in the past. These concepts, definitions and theories are of great value to practitioners, as they help us to view reality in new ways, analyse things differently, thereby gaining new insights. We also want to – concisely – explain how and why we conducted our studies, as this is what embeds them in usability practice. Our new methods and tools were not conceived out of thin air; they are the product of extensive studies of practice, and of iterative cycles of evaluating, redesigning, re-evaluating, etc. This is why we have included explanations of how the new methods and tools were validated in practice, and testimonials of how practitioners experienced applying them. We have even added some references, even though some of you indicated you did not need them. Not only is it good scientific practice, they are there for those of you interested in the theoretical background on which the methods and tools described in this book are founded. As this book is the final deliverable of the Design for Usability project, a few words of thanks are in place. First of all, to the people from Philips, Océ, T-Xchange and Indes who backed the project from the start, were willing to back that support financially, and who provided us with invaluable input and feedback. To Lilian Henze (P5 Consultants) and Rianne Valkenburg who participated very positively in the user committee. To Unilever, who came on board later, but did so with much enthusiasm and interest. To NL Agency, whose IOP-IPCR program results in valuable, practice-oriented research projects like the DfU project, and whose staff, in particular Michiel de Boer and Joop Postema, provided guidance and support. And of course,
- 7. 5Introduction Prof Daan van Eijk a big thank you to the DfU project team: Bart, Christelle, ChaJoong, Elke, Frederik, Henri, Ilse, Jasper, Mascha, Mieke, Moniek, Onno, Peter-Paul, Peter, Sonja, Stella, Steven and Tristan. Answers lead to new questions. At least, at the end of most research articles and reports you usually find suggestions for new research. We are no exception. Apart from the insights gained and the tools developed, we have identified new directions. In addition, efforts are being made to implement the UCDtoolbox, and we’re exploring whether we can continue the Design for Usability symposia as a yearly event where usability and user experience practitioners and researchers exchange the latest insights and tools for making more usable products. We would like to build on the foundations that we have created in this project, and hope that you will join us in the future. It has been a pleasure and a privilege doing this project in, for and with practitioners. Sincerely, Daan van Eijk “You are our users. The purpose of the new knowledge and methods developed is that they are to be applied by you.”
- 8. contents 6 Design for Usability Methods Tools
- 9. 1 The Design for Usability Project 8 Practice-oriented research for user-centred product design 2 Usability, interaction, experience and user-centred design 16 Learning from existing research 3 User-centred organization and process 26 3.1 Organizational barriers and enablers for usability 28 From case studies to recommendations for usability in practice 3.2 The UCD Kick-Off Tool 38 A systematic support in defining a user-centred plan of approach 3.3 UCDtoolbox.com 48 Helping practitioners explore, select and apply UCD methods 4 Anticipating usability issues 60 4.1 Preventing unawareness in usability related decision-making 62 Why asking the right questions can be more important than getting the right answers 4.2 Envisioning Use 72 A workshop technique to share use-related knowledge in product development teams 4.3 Anticipating soft problems 84 Using product properties and user characteristics to predict user (dis)satisfaction 5 Design strategies for usability 96 5.1 Guidelines to design for dynamic and diverse use situations 98 Exploring the who, where and why of product use 5.2 The Product Impact Tool 110 Designing for user-guiding and user-changing 6 Design for Usability Symposia 120 2009: Design for Usability 122 2010: Product Impact 126 2011: Methods Tools 130 7 Related projects 134 7.1 Managing Soft Reliability 136 7.2 Embedded and Embodied Usability 138 7.3 REPAR 140 Design for Usability publications 142 7
- 10. 8 Design for Usability Methods Tools Peter-Paul Steven Christelle Stella Ilse Mascha Peter Tristan Sonja Frederik Daan CJ Henri Jasper Mieke
- 11. 9 consumer products by creating new methodologies and methods for user-centred product development. Throughout the project, a considerable part of our efforts have been directed at disseminating the resulting knowledge to designers and others involved in the product development practice; we have set up the DfU website, written many ‘popular’ publications, organised symposia, and even developed animations for YouTube. Now that it’s time to wrap up the project, we can look back on an exciting journey, and are happy that the partners with whom we started this endeavour share this feeling. We believe that the efforts of the Design for Usability project have contributed to the knowledge, methods and tools that will make the electronic product development practice much more user-centred. chapter 1 The Design for Usability project Practice-oriented research for user-centred product design Daan van Eijk, Jasper van Kuijk The drivers for the Design for Usability project are what we, the designers, experience when we become involved in the development process of electronic consumer products. Aspects like the speed of development and the speed to market, complex functionality and complexity of use, are placing increasing pressure on the overall product usability. An added complexity is the rapidly increasing array of design processes, tools and methodologies available to designers and development teams. Design teams have to be aware of the existence of new, improved tools and be able to select the most appropriate tool to deal with the usability issues at stake. We believe these trends are the reason for why we, the three Dutch universities of technology, see many consumers struggle with the usability of their new electronic consumer products. With the initial support of four companies, we obtained funding from the IOP-IPCR program, and were able to setup a practice-oriented research project, taking an integrated approach towards studying usability. Our goal was to improve the usability of electronic professional and YouTube Watch a 2 minute animated summary of this introduction: http://bit.ly/dfu-summary
- 12. 10 Design for Usability Methods Tools products [10], and to make matters even worse (for design teams that is) an increasing pressure on time to market [11]. The rapid development cycles put pressure on product development activities: there is less time to perform usability tests and the recommendations made by the users during these tests cannot always be implemented [12]. In addition, implementing market feedback based on an existing product can be troublesome, because the development of a new model starts directly after the previous product design has been finalized. So, when the team starts working on the new product, its predecessor often still has to be introduced on the market [13]. Due to the increasing complexity of products, new product development requires the collaboration of multidisciplinary teams [14]. Product development teams are often distributed across the planet [12, 15] which complicates team communication [16]. Additionally, product development groups often use local subcontractors or outsource development activities which is considered detrimental for product quality [2]. Usability under pressure Many signals from the product development practice have alerted us to the fact that the usability of electronic consumer products is coming under pressure. In the past, product returns and complaints were largely due to technical failures (quality or reliability issues). Over time companies have become better and better at managing product quality and, up to the late nineties, this resulted in a decline in the number of product returns [2]. However, from that time on the number of product returns has been on the rise [13]. In a study by Den Ouden et al. [10]no technical fault could be detected in 48% of products returned by consumers. This ‘no- fault-found’ category has been estimated at 68% of returned electronic consumer products, and the cost for product returns for 2007 in the US market alone was put at $13.8 Underlying trends In this section, we review each of the drivers of the Design for Usability project in more detail. In electronic products The number of functions in electronic products has been steadily increasing for a number of years [1-3]; this is a result of both the advances in technology [4, 5] and the commercial advantage of offering innovative and increasing functionality [6]. At the same time, many electronic products are getting smaller and smaller [3] Products with a more elaborate functionality are generally harder to use than those with a limited number of functions [7, 8] and this becomes an even greater problem when all these functions have to be accessed through a small user interface [9]. In product development practice Along with the increase in functionality, we have seen an increase in the technological complexity of electronic Figure 1: Increasing functionality, decreasing dimensions, and increased networking results in a decrease in the ‘guessability’ of music players.
- 13. 11chapter 1 The Design for Usability project Product impact: improve understanding of how users change in the process of interaction with products, and to integrate this knowledge in design practice, by means of a Product Impact Tool. Two additional topics were tackled by other members of the project team: Envisioning use: devising a technique which helps teams create a common vision on product use at an early phase of product development by team alignment and sharing information with regards to future use situations; Dynamic and diverse use situations: gaining insights into how designers deal with designing products that are used in continually changing (dynamic) or different (diverse) contexts of use, and translating these insights into guidelines. Participating parties The Design for Usability project was Initiated and executed by: Delft University of Technology Eindhoven University of Technology University of Twente In cooperation with and supported financially by: Indes Océ Technologies Philips T-Xchange billion [17]. This return of products, even though technically speaking they are not broken, can partly be attributed to people not understanding how to use a product properly and therefore concluding that it does not work, as well as to those consumers dissatisfied with the product because it did not meet their expectations [10]. Improving the usability of products is seen as one of the strategies to deal with this rise in returns [17]. The project Goal The Design for Usability project aims to improve the usability of electronic professional and consumer products by creating new methodologies and methods for user-centred product development which can be quickly and easily applied in design practice. Work packages Five primary themes formed the basis of the DfU project, each of these themes was investigated as part of a PhD project: Design methodology: developing a project kick-off tool that supports product development teams in specifying a detailed user-centred plan of approach; Barriers and enablers for usability in practice: identifying which properties of product development organizations influence usability; Usability decisions in design practice: identifying critical factors that influence the quality of usability-related decision-making and that may cause teams to overlook or misjudge usability problems; User characteristics, product type, and soft usability problems: identifying which combinations of personality traits and product types lead to usability problems that trigger user dissatisfaction;
- 14. 12 Design for Usability Methods Tools T-Xchange Unilever P5 consultants Rianne Valkenburg NL Agency In addition to the team above, the project was made possible by: Elke Den Ouden: chair of the user committee (before 2010) Onno van der Veen: chair of the user committee (from 2010) Bart Ahsmann (TU Delft): consultant on valorisation issues Approach Integration of user-centred design The Design for Usability project team took an integrated approach to user-centred design (UCD). We studied product development processes, teams and organizations, rather than just investigating how user tests were conducted or how usability departments are organized. We were interested in discovering how the principles of user-centred design are integrated in the product development process. Practice-oriented The aim of the Design for Usability research project, improving how usability is dealt with in ‘real life’ product development, necessitated a practice-oriented research approach. At the same time as our initial exploration of the topic by reviewing the literature, interviews were conducted with usability practitioners and experts. Secondly, a major part of data collection was achieved by working on case studies [18] in product development practice. Thirdly, many users were involved by means of survey studies and experiments. Finally, the DfU project involved regular ‘member checks’ [19]: throughout the studies, specialists verified any interpretations and conclusions, and each of the Funded by: The Netherlands Ministry of Economic Affairs, Agriculture and Innovation through NL Agency under the IOP-IPCR program. Team The project Design for Usability was a collaboration between a large group of design and development specialists. The core research projects were performed by five PhD candidates. Additional researchers from the three universities were involved in the project, working on smaller but related research projects. PhD candidates Steven Dorrestijn Christelle Harkema Frederik Hoolhorst Cha Joong Kim Jasper van Kuijk Researchers Stella Boess Mieke van der Bijl-Brouwer Henri Christiaans Peter Sonnemans Ilse Luyk-de Visser Peter-Paul Verbeek Mascha van der Voort Tristan Weevers Project coordination Daan van Eijk (chairman) Sonja van Grinsven User committee Indes Océ Philips
- 15. 13chapter 1 The Design for Usability project the results to the design research community, and which also allows for the education of a new generation of designers and product developers. sub-projects included at least one feedback workshop in which the results and conclusions were discussed with practitioners. Results Outcomes The Design for Usability project has resulted in: 1 A reference methodology for organising product development processes and organizations if the goal is to make usable products; 2 New methods and tools for user-centred design, and 3 New insights in issues obstructing and facilitating the creation of usable products in product development practice. Dissemination to practice As the project’s goal is to improve product development practice, in addition to scientific publications, we made great efforts to communicate our project results to practitioners through: Continuous exchange of information with the companies involved in the project, for example through workshops and presentations; Giving presentations and interviews; An up to date, content-rich and appealing website; Presenting research results not only in articles, but also in formats more accessible to practitioners, such as manuals, card sets, booklets, etc. Three Design for Usability symposia on World Usability Day (2009/2010/2011); This Design for Usability ‘Methods Tools’ book, supported by content on the project website. In addition, the results were published in PhD theses and scientific journals (see ‘Design for Usability Publications’ at the back of the book), which ensures dissemination of Figure 2: The Design for Usability team during a retreat in Arnhem
- 16. 14 Design for Usability Methods Tools v. d. Aalst, H. Corporaal, J. B. O. S. Martens, P. H. A. v. d. Putten, A. J. M. M. Weijters, and A. C. Brombacher, “Soft Reliability: An Interdisciplinary Approach with a User–System Focus,” Quality and Reliability Engineering International, vol. 25, pp. 3-20, 2008. [5] D. A. Norman, The Design of Everyday Things. New York: Basic Books Publishers, 2002. [6] J. M. Thölke, E. J. Hultink, and H. S. J. Robben, “Launching new product features: a multiple case examination “ Journal of Product Innovation Management, vol. Volume 18, pp. 3-14, 2001. [7] J. Keijzers, E. den Ouden, and Y. Lu, “Usability References [1] D. A. Norman, “Three challenges for design,” Interactions, vol. 14, pp. 46-47, 2007. [2] E. Den Ouden, “Development of a design analysis model for consumer complaints : revealing a new class of quality failures “ PhD thesis, Eindhoven University of Technology, Eindhoven, 2006. [3] C. Lindholm, T. Keinonen, and H. Kiljander, Eds., Mobile Usability. How Nokia changed the face of the mobile phone. McGraw-Hill, 2003. [4] A. Koca, M. Funk, E. Karapanos, A. Rozinat, W. M. P. Figure 3: In the Design for Usability project, manuals, card sets, and booklets were used to disseminate knowledge, methods and tools to practitioners.
- 17. 15chapter 1 The Design for Usability project Marketing Perspectives,” Journal of Product Innovation Management, vol. 14, pp. 35-47, 1997. [17] T. Steger, B. Sprague, and D. Douthit, “Big Trouble with No Trouble Found: How Consumer Electronics Firms Confront the High Cost of Customer Returns,” Accenture, 2007. [18] R. K. Yin, Case study research: design and methods, 4th edition ed. Thousand Oaks, California, USA: SAGE publications, 2009. [19] A. K. Shenton, “Strategies for ensuring trustworthiness in qualitative research projects,” Education for Information, vol. 22, pp. 63-75, 2004. benchmark study of commercially available smart phones: cell phone type platform, PDA type platform and PC type platform,” presented at 10th international conference on Human computer interaction with mobile devices and services, Amsterdam, The Netherlands, 2008. [8] R. T. Rust, D. V. Thompson, and R. W. Hamilton, “Defeating feature fatigue,” Harvard Business Review, vol. 84, pp. 98-107, 2006. [9] T. Keinonen, “One-dimensional usability - Influence of usability on consumers’ product preference,” Smart Products Research Group, University of Art and Design, Helsinki, 1998. [10] E. Den Ouden, L. Yuan, P. J. M. Sonnemans, and A. C. Brombacher, “Quality and Reliability Problems from a Consumer’s Perspective: an Increasing Problem Overlooked by Businesses?,” Quality and Reliability Engineering International, vol. 22, pp. 821-838, 2006. [11] A. C. Brombacher, P. C. Sander, P. J. M. Sonnemans, and J. L. Rouvroye, “Managing product reliability in business processes ‘under pressure’,” Reliability Engineering and System Safety, vol. 88, pp. 137–146, 2005. [12] S. Minderhoud and P. Fraser, “Shifting paradigms of product development in fast and dynamic markets,” Reliability Engineering System Safety, vol. 88, pp. 127- 135, 2005. [13] A. C. Brombacher, “Reliability in strongly innovative products; a threat or a challenge?,” Reliability Engineering System Safety, vol. 88, pp. 125-125, 2005. [14] M. Kleinsmann, “Understanding Collaborative Design,” PhD thesis, Delft University of Technology, Delft, 2006. [15] P. Ketola, “Integrating Usability with Concurrent Engineering in Mobile Phone Development,” PhD thesis, University of Tampere, Tampere, Finland, 2002. [16] X. M. Song, M. M. Montoya-Weiss, and J. B. Schmidt, “Antecedents and Consequences of Cross-Functional Cooperation: A Comparison of RD, Manufacturing, and
- 18. 16 Design for Usability Methods Tools
- 19. 17 chapter 2 Interaction, experience, usability and user-centred design Learning from existing research Jasper van Kuijk The speed dial on this jigsaw incorporates knowledge that most users don’t have, namely what the frequency of the saw blade should be for each material. A frame of reference Before presenting the methods and tools that were developed within the Design for Usability project, we want to provide a frame of reference. A great deal of theory and method development in the field of user-centred design has been accomplished previously, so we are not about to let that go to waste. Secondly, having a shared view on concepts like interaction, experience, and, of course, usability, is essential for effective communication of the results of the project. So below we discuss how we defined electronic consumer products, and how usability, interaction and experience relate. Lastly, we discuss the properties of the approach that has the intention of creating usable products: user-centred design. Electronic products The Design for Usability project focused on the usability of electronic products for the professional and consumer domain. These are products with a physical presence (as opposed to software), and that feature a certain amount of integrated information technology that enables them to interact with the user (as opposed to, e.g., chairs and vases). Some examples are mobile phones, microwaves, office printers and MRI scanners. Differences with the digital domain Important differences between electronic products and the digital-only domain (such as ICT systems, websites and software) are: Physical presence: the usability of electronic products is not only influenced by the on-screen user interface, but also by the design of other input and output modalities, and the embodiment of the product; Unique platforms: digital products usually run on more standardized technological platforms (servers, operating systems, motherboards, etc.) than electronic products, for which often custom-built hardware has to be developed; Design/development characteristics: when developing software it is more common to have design and implementation coincide; the product is designed while being programmed [1]. Secondly, in the development
- 20. 18 Design for Usability Methods Tools of digital products there is no clear moment when the design is more or less ‘frozen’ because investments have been made in moulds and orders for parts have been placed. Ecosystems Electronic products increasingly function as a part of a network of products and services, referred to as the ‘ecosystem’ [1-3] (Figure 1). For example a television can be part of a ‘network’ with a home cinema set, a hard disk recorder and a satellite decoder. Even if the usability of the individual products is acceptable, this does not guarantee that the system as a whole is usable. In product ecosystems three sub-categories can be distinguished [4]: 1 The core product: what the user primarily interacts with; 2 The extended product: those parts that facilitate the use of the core product; 3 Symbiotic products, software, services and content that allows the core product to function. User interface The user interface (UI) consists of those parts of the product which enable users to use it for its intended purpose [5]. It includes all elements of a product-service system that allow people to control it and receive information on how to interact with it. Examples of elements of the UI are, for example, the display and on-screen UI, controls and audio output components). But also the embodiment of a product can be considered part of the user interface, as users may also attribute a certain meaning to the appearance of the product, for example which product category it falls into and how it (thus) should be operated [6], and, because the embodiment influences how the product can be interacted with physically, the physical (dis)comfort users experience [7]. What is usability? Most of us have at one time faced sales people who tell us, ‘This product does this and that’. This is fine, but what is most relevant is what you can do with the product. As a common mantra in user-centred design goes, ‘If the user can’t find it, it’s not there.’ When talking about usability, we take an instrumental view on products. We are not interested in whether you want to buy them, own them or caress them: we are focused on whether you can use Figure 1: Visualization of the primary components of electronic consumer products, clustered by the core product, extended product and symbiotic components.
- 21. 19chapter 2 Interaction, experience, usability and user-centred design them. In other words, whether you at least can do what its maker intended you to do with the product, and maybe more. Defining usability The most commonly used definition of usability is the one by the ISO organization [8]: “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” In which the three main dimensions of usability are defined as: Effectiveness: accuracy and completeness with which users achieve specified goals. Efficiency: resources expended in relation to the accuracy towards the use of the product. E.g., required time or mental effort. Satisfaction: freedom from discomfort, and positive attitudes towards the use of the product. These three dimensions are quite generic. They define what makes a product usable at a conceptual level: whether users can reach their goals, at what cost, and how they feel about using the product. However, these dimensions are not very specific. Depending on the (type of) product, it still needs to be specified how to operationalize them, which measures can be used, and what the essential goals per phase of use are [9]. In different phases of product use, different ‘types’ of usability can be important, e.g., guessability, learnability or experienced user performance (Figure 2). Usability and functionality The goals that a product can help the user achieve, such as cleaning clothes or playing music, are referred to as its functionality or utility [10, 11]. The ISO definition of usability refers to the extent to which users can apply the product to reach their goals. This implies that usability is not only about whether people can access the functionality the product offers, but also about whether the product offers the right functionality. Take for example the usability of E-books versus the usability of conventional books. For the goal of quickly looking up a page and scanning the content, conventional books can be considered to be more efficient, whereas for the goal of being able to read during your holiday, an eBook may be a more effective and efficient solution. Time per task Experience of task (task expertise) Relearnability Gulf System potential EUP (Experienced User Performance) Learnability Gulf Guessability Gulf Figure 2: Performance curve with components of usability as specified by Jordan [1]. The time required to complete tasks is displayed as a function of how long users have used the product. First people have to familiarize themselves with the product and learn how to operate it (‘guessability gulf’), followed by a first phase of use during which people learn how to use the product (‘learnability gulf’). The ‘relearnability gulf’ refers to how repeat use during time per task is influenced by the ease with which product use can be recalled. And finally, as users gain more and more experience, time per task converges on ‘experienced user performance’.
- 22. 20 Design for Usability Methods Tools Human-product interaction is a complex interplay of the components of the context of use: the users, tasks, product, and its environment [8]. Based on existing frameworks [5, 10] within the DfU project, we developed a new framework for human-product interaction specifically with electronic products in mind [4]. It consists of the following elements: Product-service combination: what the user applies to reach a goal; Symbiotic products: to stress that electronic consumer products increasingly function in networks of products; User(s): who interact(s) with the product. Includes the goals (intended outcomes) they have, as well as characteristics like knowledge, skill, experience, education, training, physical attributes, and motor and sensory capabilities; Other people: people who do not directly interact with the product-service system, but who can affect and be Human-product interaction From the ISO definition it can be deduced that usability includes both user performance (effectiveness and efficiency) and user experience (satisfaction about use). User performance can be observed or measured by looking at human-product interaction, which is defined as [5]: Bi-directional information exchange between users and equipment, which may include physical actions, resulting in sensory feedback. We can distinguish between using a product and interacting with it. Using a product is the application of a product in order to reach a goal [12] - for example, we can use a vase to hold water and flowers. Interacting with the aforementioned vase would be for example filling it with water and putting the flowers in it. Figure 4: Framework for human-product interaction. Figure 3: Left: Using a vase for the purpose of holding flowers and providing them with water. Right: interacting with a vase with the purpose of putting water in the vase.
- 23. 21chapter 2 Interaction, experience, usability and user-centred design affected by the use of a product; Environment: includes the physical environment (e.g. workplace, furniture), the ambient environment (e.g. temperature, humidity) and the social and cultural environment (e.g. work practices, organizational structure and attitudes). Context-dependent The properties of components of human-product interaction can vary considerably. For example, the interaction that takes place with a phone when a teenager is running to catch a train while making a phone call is completely different than that of a 45-year old man sitting at a kitchen table and sending a text message. Human-product interaction, and as a consequence usability, is determined by who uses a product, what they are trying to achieve, the other products in the network, other people present, and the environment in which the product is used. Usability is defined as: ‘The extent to which a product can be used by specified users to achieve specified goals (…) in a specified context of use.’ In other words: there is no such thing as the usability of a product, as usability depends on the context of use [8]. User diversity User characteristics that can influence human-product interaction include knowledge, skill, experience, education, training, physical attributes, and motor and sensory capabilities[8]. These can vary greatly from one user to the next and thus usability can vary greatly between user groups. But even when observing just one person, we may observe great differences from one day – or even from one moment – to the next; even though bodily measures and cognitive capabilities remain the same, the patterns of use of individual users can vary greatly [13, 14]. Diversity in user goals An important aspect which determines the usability of a Figure 6: The relation between human-product interaction, user experience and the ISO definition of usability. Two out of three dimensions of usability (effectiveness and efficiency) relate to human-product interaction. The third (satisfaction) is about how users experience that interaction (user experience). Figure 5: Visualization of the relation between human-product interaction (black, left) and the user experience (right, red). The user experience is the user’s affective response to the interaction, which can iteratively influence the interaction.
- 24. 22 Design for Usability Methods Tools Usability: user performance and user experience If we look at the dimensions of usability, we can distinguish between ‘user performance’ measures (effectiveness and efficiency) and subjective assessments of ease of use (satisfaction) [10]. We can state that the first two measures refer to human-product interaction (what happens) and the third refers to user experience aspects (how people experience what happens). Attention needs to be paid to both aspects, because the relation between the two cannot always be predicted. In some cases the user performance dimensions can be more important, while in other cases user experience aspects should be prioritized. Because satisfaction is influenced by expectations [23] it could, for example, be that a person who owns a particularly unusable MP3 player is very satisfied about the effectiveness and efficiency of this MP3 player, and that others would consider this to be mediocre, because they previously owned a very usable model. User-centred design Creating products and product-service systems with a high level of usability can be achieved by applying a user-centred design approach. The designer-user gap Because users and situations in which products are used can vary greatly, as a designer it is hard to predict all the situations that users may use a product in. First of all, because the designer may differ greatly from the person he or she is designing for. Nielsen [24] refers to this as the ‘designer-user gap’. Secondly, the people who worked on the product know much more about the product than the people who will be using it, because they – after all – designed it, often from the ground up. So they know all the intricacies, all the menus, all the small tricks, that there is a button at product is the goal that the user is trying to reach at that time. The goal may vary with the different stages of product use [15, 16]; users have a different goal when they are trying to install a product, when first using a product, or when using a product for the millionth time. User goals are also influenced by the role of the user. The user of a system can be to the so-called ‘end-user’, but he or she can also have other roles. For example, for an office printer and copier, user groups include the ‘end-users’ (who get coffee), service- people (who refill the cartridges), and maintenance staff (who install and maintain the product) [17]. But even within the end-user group there can be a considerable variety of user types. For example, a department secretary will use a printer/copier in a different way than an office worker. User Experience The term ‘user experience’ has been widely used to describe what using a product ‘does’ to the user. It is often billed as the ‘next step beyond usability’; products should be usable, but they should also appeal to people’s sense of aesthetics, provide pleasure, etc. To define user experience we have adapted a definition made by Forlizzi Battarbee [18]: The constant stream of ‘self-talk’ that happens as a consequence of being exposed to, applying and interacting with products; how we constantly assess our goals relative to the products we observe and use. Interaction is what happens, user experience is how that makes us feel (Figure 4). The fact that it includes an evaluative act seems to make user experience related to the notion of satisfaction: whether a product lives up to the expectations that people have about a product [19, 20]. This implies that the user experience can also be influenced by factors such as advertising, word-of-mouth or product appearance [21, 22].
- 25. 23chapter 2 Interaction, experience, usability and user-centred design the back you can push, that you need to connect this cable before that cable, etc. It is difficult to ‘unlearn’ all that. Principles of user-centred design User-centred design helps to compensate for these issues: the designer-user gap and the design team’s ‘unlearnable’ knowledge about the product’s workings. User-centred design can be described as [25]: “An approach in which ‘product quality should be measured from a user point of view, taking into account needs, wishes, characteristics and abilities of the projected user group” The ISO standard on Human-centred1 Design for Interactive Systems [26] provides six basic principles: 1 The design is based upon an explicit understanding of users, tasks and environments; 2 Users are involved throughout design and development; 3 The design is driven and refined by user-centred evaluation; 4 The process is iterative; 5 The design addresses the whole user experience; 6 The design team includes multidisciplinary skills and perspectives. User involvement and representation Basically, user-centred design brings an extra perspective to the development process: that a product does not work if users do not understand how it works. To make sure that users understand the product, designers need to bridge the designer-user gap by adding user-involvement to design. 1 Some prefer the term ‘human-centered design’ as they believe this does justice to view that people are more than users of products. User involvement is the act of collecting information about users or from users [27, 28]. In some cases, conducting a simulation with actual users is not possible, but product developers still need to anticipate product usage, in which case they might resort to so-called ‘inspection methods’ [29], such as an expert evaluation or a method that methodically simulates product use. As a counterpart to user involvement, we refer to these methods as ‘user representation’
- 26. 24 Design for Usability Methods Tools [10] B. Shackel, “The concept of usability,” in Visual Display Terminals, J. Bennet., Ed., ed Englewood Cliffs, New Jersey: Prentice-Hall, Inc., 1984, pp. 45-81. [11] J. Grudin, “Utility and usability: research issues and development contexts,” Interacting with Computers, vol. 4, pp. 209-217, 1992/8 1992. [12] N. F. M. Roozenburg and J. Eekels, Product Design: Fundamentals and Methods. Chichester: John Wiley Sons, 1995. [13] H. Kanis, “Usage centred research for everyday product design,” Applied Ergonomics, vol. 29, pp. 75- 82, 1998/2 1998. [14] M. Weegels, “Accidents Involving Consumer Products,” PhD thesis, Delft University of Technology, Delft, 1996. [15] P. W. Jordan, “What is usability?,” presented at Contemporary Ergonomics, 1994. [16] N. F. M. Roozenburg and J. Eekels, Produktontwerpen, structuur en methoden. Utrecht: Uitgevery Lemma, 1991, 1991. [17] J. Nielsen, “The usability engineering life cycle,” IEEE Computer, vol. 25, pp. 12-22, 1992. [18] J. Forlizzi and K. Battarbee, “Understanding experience in interactive systems,” in 2004 conference on Designing interactive systems: processes, practices, methods, and techniques, Vienna, Austria, 2004, pp. 261-268. [19] K. Matzler, H. H. Hinterhuber, F. Bailom, and E. Sauerwein, “How to delight your customers,” Journal of Product and Brand Management, vol. 5, pp. 6-18, February 1996 1996. [20] F. F. Reichheld, “The one number you need to grow,” Harvard Business Review, vol. 81, pp. 46-55, Dec 2003. [21] S. L. Wood and C. P. Moreau, “From Fear to Loathing? How Emotion Influences the Evaluation and Early Use of Innovations.,” Journal of Marketing, vol. 70, pp. 44- 57, 2006. References [1] B. Buxton, Sketching User Experiences: Getting the Design Right and the Right Design: Morgan Kaufmann, 2007. [2] I. M. De Visser, “Analyzing user perceived failure severity in consumer electronics products : incorporating the user perspective into the development process “ PhD Thesis, TU Eindhoven, Eindhoven, 2008. [3] E. L.-C. Law, V. Roto, M. Hassenzahl, A. P. O. S. Vermeeren, and J. Kort, “Understanding, scoping and defining user experience: a survey approach,” presented at the 27th international conference on Human factors in computing systems, Boston, MA, USA, 2009. [4] J. I. van Kuijk, “Managing Product Usability; How companies deal with usability in the development of electronic consumer products,” PhD thesis, Delft University of Technology, Delft, 2010. [5] ISO, “ISO 20282-3: Ease of operation of everyday products -- Part 3: Test method for consumer products,” International Organization for Standardization, Geneva, Switzerland2007. [6] S. Boess and H. Kanis, “Meaning in product use: A design perspective,” in Product Experience, H. N. J. Schifferstein and P. Hekkert, Eds., Reprint Edition, 2009 ed Oxford, UK: Elsevier, 2008. [7] P. Vink, Ed., Comfort and Design: principles and good practice. Boca Raton, Florida, USA: CRC Press, 2005. [8] ISO, “ISO 9241-11 Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability,” International Organization for Standardization, Geneva, Switzerland ISO 9241- 11:1998(E), 1998. [9] N. Bevan and M. Macleod, “Usability measurement in context,” Behaviour Information Technology, vol. 13, pp. 132-245, 1994.
- 27. 25chapter 2 Interaction, experience, usability and user-centred design [22] E. Karapanos, J. Zimmerman, J. Forlizzi, and J.-B. Martens, “User Experience Over Time: An Initial Framework,” presented at the CHI 2009, Boston, Massachusetts, USA, 2009. [23] G. Lindgaard and C. Dudek, “What is this evasive beast we call user satisfaction?,” Interacting with Computers, vol. 15, pp. 429-452, 2003/6 2003. [24] J. Nielsen. (2008, May 30). Bridging the Designer- User Gap. Available: http://www.useit.com/alertbox/ designer-user-differences.html [25] K. Vredenburg, S. Isensee, and C. Righi, User-Centered Design; An Integrated Approach. New Jersey, USA: Prentice-Hall, 2002. [26] ISO, “ISO 9241 Ergonomics of human-system interaction,” in Part 210: Human-centred design for interactive systems (formerly known as 13407). ed. Switzerland: ISO, 2010. [27] J. Grudin, “Obstacles to user involvement in software product development, with implications for CSCW,” International Journal of Man-Machine Studies, pp. 435-452, 1991. [28] S. Lauesen, “Usability Engineering in Industrial Practice,” in INTERACT’97, Discovering New worlds of HCI, Sydney, Australia, 1997, pp. 15-22. [29] T. Rooden, “Design models for anticipating future usage,” PhD thesis, Applied Ergonomics and Design, Delft University of Technology, Delft, 2001.
- 28. chapter 3 User-centred organization and process 26 Design for Usability Methods Tools
- 29. One of the main goals of the Design for Usability project was to develop a coherent methodology, an integrated process architecture, for user-centred design projects. There was a great need for this, because although there are a large number of individual UCD methods available for user research and usage evaluation, there is little information on what integrated, user-centred product innovation should look like. This chapter outlines projects and results that were set up to achieve the main DfU goal. The approaches included the following three perspectives: how to organize a product development group, planning a product development project, and the selection of appropriate methods to apply within a project. 3.1 Organizational barriers and enablers for usability From case studies to recommendations for usability in practice Jasper van Kuijk studied how companies deal with usability in the development of electronic consumer products, which resulted in the identification of organizational barriers and enablers for usability in practice. Based on this he wrote 25 recommendations on how to organize a product development group if the goal is to make usable products. 3.2 The UCD Kick-Off Tool A systematic support in defining a user-centred plan of approach From the organization of a development group as a whole to the organization of individual projects: Frederik Hoolhorst investigated how to successfully plan user-centred design and development projects. Based on interviews with practitioners, case studies and product development literature, he conceived the UCD Kick-Off Tool which helps organizations to setup UCD-projects. 3.3 UCDtoolbox.com Helping practitioners explore, select and apply UCD methods One of the steps in the UCD Kick-Off Tool is the identification of appropriate methods for user- centred design and then planning how to execute them. Tristan Weevers developed UCDtoolbox. com, an online resource for exploring, selecting and learning about methods for user-centred design. 27
- 30. 28 Design for Usability Methods Tools
- 31. 29 an overview of mechanisms of barriers and enablers for usability, and the identification of four primary drivers for usability. The overall conclusion was that for a user-centred product development process to take place, the complete organization needs to be user-centred. The findings were used in 25 recommendations for industry on how best to organize product development if the goal is to make usable electronic consumer products. 3.1 Organizational barriers and enablers for usability From case studies to recommendations for usability in practice Jasper van Kuijk The aim of this project was to identify which factors in product development have a positive or negative influence on usability Summary Over time, many methodologies and methods for user- centred product development have been developed, but as consumers and companies are still faced with products with poor usability, it is worthwhile investigating whether and how these methods are applied in product development practice. To gain insights into how usability is dealt with in the development of electronic consumer products, three case studies were conducted: 1) in four adjacent sectors, 2) at five development groups, 3) in three development projects at one development group. Interviews were conducted with 69 product developers across 10 product development groups. The results provide a description of how usability is dealt with in product development of electronic consumer products, YouTube Watch a 2½ minute animated summary of this research: http://bit.ly/be-summary
- 32. 30 Design for Usability Methods Tools return the queue is gone. I use the ATM and withdraw 300 Euros. The ATM returns my card, but no money comes out. Now I start worrying as well. I decide to try again, assuming, hoping that the bank will not process the transaction if no money came out. As I look down to read the instructions, this time I notice the text at the very bottom of the opening screen (that disappears as soon as you slide your card into the slot): ‘This machine will not distribute sums greater than 250 Euros.’ People worried. People got annoyed. A queue formed at the machine and inside the bank. I think its fair to say that the customer experience of this bank suffered. But what this situation made me wonder most is: why is this ATM so unclear? Why just a statement on the opening screen? Why not a clear warning if a transaction cannot be processed. Why not simply not allow people to ask for an amount higher than 250 Euro. Basically, it makes me wonder: what went wrong during development. Did the designers not know about interaction design principles? Was this use case not involved in user testing? Was any user testing performed, for that matter? These are the type questions that I answered when working on my PhD. The aim of my PhD project was to identify barriers and enablers for usability in product development practice of electronic consumer products ( the story above about the ATM is an example from professional electronics). The focus was on electronic consumer products (e.g., portable music players, washing machines and mobile phones), because in this sector usability is under pressure from increasing product complexity, commoditization and speed of development. In a discussion of research in the medical sciences, Malterud [1] argues that in addition to controlled experiments, with their focus on questions and phenomena that can be Introduction A couple in their late fifties is blocking the only ATM machine in Budapest that spits out Euros. And I need Euros. The couple turns out to be Australian. The man tried to withdraw 500 euros from the ATM, got his card back, but no money. In a display of utter innocence the machine has returned to its default state, showing just the welcome screen. The man says he will go into the bank and ask what the couple should do. The couple decides the woman will stay at the ATM in case the money all of a sudden does turn up, and to prevent other people from using the ATM, because they don’t want to change the state of the machine. Perhaps the people from the bank can find a way to get their money out. A queue starts forming. Embarrassed, the woman tells the people in line what happened, and why they cannot use the machine. As an alternative people start walking into the bank, where the Australian man is still waiting for his number to be called. More and more people come into the bank, hoping to get Euros at a cash desk. I leave. When, half an hour later, I Author Bio Jasper van Kuijk is Assistant Professor at TU Delft with a focus on the practice of user- centred innovation. He obtained a Master’s degree in Industrial Design Engineering (IDE) at TU Delft, specializing in human- product interaction of electronic consumer products. He then worked two years as a design management consultant before starting his PhD in the DfU project.
- 33. 313.1 Organizational barriers and enablers for usability controlled, measured and counted, the knowledge of experienced practitioners should be studied, because this could offer a broader understanding of a phenomenon. The same is true for product development. Because experienced product developers have been immersed in product development on a daily basis, they can possess a wealth of knowledge on what does and does not work for usability in product development practice. This research project was set up to tap into that knowledge. Research method In a case study on human-centred design (HCD) in the development of ICT systems, Steen [2] observed that human-centred design practice is very different from HCD principles and theory. Having worked both in commercial and academic environments, Dennis Wixon [3] and Donald Norman [4] underline the contrast between usability in research and in practice: real, day-to-day product development is messy (at best), and that not only should the workings of methods for user-centred design when they are applied in a controlled setting be studied, but also what happens when they are used in the fast-paced, hectic process of developing electronic consumer products. This research project consisted of three case studies. The case study is a suitable method for explanatory studies into ‘a contemporary set of events over which the investigator has little or no control’ [5]. The first study was interview- based, and its goal was to explore how usability is dealt with in four sectors ‘adjacent’ to the electronic consumer products market (e.g., automotive, office coffee machines). Secondly, an interview-based case study was conducted at five major international product development groups in the electronic consumer products sector. The goal of this study was to identify barriers and enablers for usability in practice. The third and final case study investigated the development history of three electronic consumer products within one product development group. This resulted in a detailed Case study I 4 markets, 4 companies Fast Moving Consumer Goods Office Coffee Machines Office Copiers Automotive Case study II CE market, 5 companies Fast Moving Consumer Goods Office Coffee Machines Office Copiers Automotive Case study III One company, 3 projects conceptual framework conclusions, discussion recommendations recommendations for industry dissemination literature survey exploratory interviews Fast Moving Consumer Goods Office Coffee Machines Office Copiers Automotive Electronic Consumer Products Figure 1: The setup of this research project
- 34. 32 Design for Usability Methods Tools on the skills and attitude of the team that performs the process, how the project is set up, and the organization within which the project is executed. This in turn is influenced by the type of market a company operates in. Making usable products is about more than user research and usage evaluation. It is also about having the freedom and resources to something with the outcome of the evaluations and user tests; it is about knowing which technologies might help resolve a usability issue or about prioritizing usability in tough compromises, and about convincing a whole team that the design of a product should be changed. In other words: making usable products requires an integrated, organizational approach. Recommendations for usability in practice Based on insights from my research and from existing literature on usability in practice, I developed 25 recommendations on how to organize a company where description of how the product development group dealt with usability as well as two explanatory models of how usability is dealt with in product development. In all case studies, we took an integrated approach: the focus was on the product development process as a whole (as opposed to design only), and not just on the usability specialist and interaction designer, but on the six roles that were considered to have most influence on usability: the product manager, marketing specialist, industrial designer, interaction designer, usability specialist and development engineer. Interviews were conducted with 69 product developers across 10 product development groups. Results The case studies showed that the process of product development is only partly determined by the ‘official’, documented and/or prescribed processes. It also depends Figure 3. The recommendation card setFigure 2: Acting out the six roles in the development of electronic consumer products: the product manager, marketing specialist, industrial designer, interaction designer, usability specialist, and development engineer.
- 35. 333.1 Organizational barriers and enablers for usability the goal is to make usable products. The recommendations provide concrete guidelines for practitioners, but also includes the underlying, more abstract principles. Most of the recommendations were not conceived by The author, but were encountered through the case studies and in the literature. Together they represent ‘best practices’; they are ‘how I would do it’ if I were to organize a product development group that had to make usable electronic consumer products. Content The recommendations range from pragmatic and easily applicable (e.g., use guerrilla HCI techniques) to high-level and challenging (e.g., align the organization with user needs). As a consequence, the target audience might differ per recommendation: upper management, product managers, managers of product development teams, interaction and product designers and – of course – usability specialists. The recommendations are grouped according to a categorization scheme developed in the DfU project (Figure 4) that shows its primary investigative domains and their relations: Usability 101: how to define usability and assess its consequences? Process: what does a user-centred product development process look like; which methods should be applied, and how? Team: how to assemble a team that is capable of executing a user-centred product development process? Project: how to organize, facilitate and plan user-centred product development? Company: how to organize a company so that it facilitates user-centred product development? Market (or Context): what are appropriate retail and marketing strategies for companies that make usable products? Figure 4: The primary investigative domains of the DfU project which formed the basis for the categorization of the recommendations. 12 Design of the cards The build-up of the cards (Figure 5) includes a (provocative) title supported by an illustration, a summary of benefits and requirements for acting on the recommendation, and finally an elaborate explanation of the recommendation itself. For a tools/methods card set the text is (relatively) lengthy. This was done on purpose, because the objective was for practitioners to be able to act based on the information provided, and not just to raise awareness, which is what was observed to happen when less information was provided. Workshop To promote dissemination and implementation of the recommendations, and to collect feedback for future versions, a workshop was developed in which participants engage in a discussion about the recommendations. An effective group composition seems to be four to five representatives from different disciplines (e.g., interaction
- 36. 34 Design for Usability Methods Tools design, usability specialist, product manager, upper management) from one company. Before the workshop, each participant reads the card set and selects their three most and least favourite recommendations and provides a motivation for selecting them. During the workshop multiple discussion rounds are held according to the following steps: 1 One participant introduces a most (or least) relevant recommendation and provides motivation in max. 3 minutes [facilitator probes for concrete examples, argumentation, perceived benefits, potential roadblocks]. 2 The facilitator asks whether other group members have chosen the same recommendation (as favourite or non-favourite). If so: provide opportunity for additional motivation (also max. 3 minutes). 3 Rest of the group responds from their perspective (max 1. minute pp.). The facilitator probes for agreement, disagreement, motivations, roadblocks, and benefits. 4 Move on to next participant, next recommendation. At the end of the workshop the group tries to come to a consensus on what they believe to be the three most relevant and three least relevant recommendations. If the workshop consisted of multiple groups, this selection is presented at a plenary session, together with the argumentation. Benefits The case studies provide design researchers with a situation comparable to case studies they might conduct, and the insights they need to develop ‘designer-centred’ tools and methods for user-centred design. The recommendations for usability in practice provide actionable information on how to setup a user-centred product development organization. Conducting a workshop with the card set can be a starting point for an action plan towards a more user-centred organization. It brings to light the differences in viewsFigure 5: Explanation of the layout of the cards
- 37. 353.1 Organizational barriers and enablers for usability the recommendations on my weblog1 and invited product development professionals and design researchers to provide feedback, an example of which is given below. between disciplines which may facilitate understanding and change. Because the workshop setup is simple and the card set is relatively rich in content, the workshop can be executed without an external facilitator. Validation in practice Apart from product development practice being the focus of this research, I adopted what one could label a ‘practitioner-centred’ research approach. While exploring the topic, in addition to conducting a literature survey, I conducted interviews with usability practitioners and experts. Throughout the studies, product development practitioners were treated as informants, not as the subject of study. In addition, throughout the case studies I verified whether informants found my interpretations and conclusions accurate and complete: each of the case studies included a feedback workshop or workshops in which the results and conclusions were presented and discussed. To assess to what extent the recommendations made sense and were relevant to product developers I published Wietske Rodenhuis - Product Marketing Director at Philips Lighting On having your business group be the subject in a case study Figure 6: The card set in use during a workshop facilitated by the Institute of Design Knowledge in Hong Kong (Photo by HOPF images, © Institute of Design Knowledge) “Jasper conducted one of his case studies within the business group where I was product manager at the time. I found it a great opportunity to ensure that insights from product development practice find their way into academic research; sometimes I have the idea that this does not happen enough. On the other hand, it was also quite an interesting experience to have someone in your organization who asks questions about things that you consider completely normal and reflects critically on how you approach day-to-day product development. However, the most valuable element in his case study to me was the cross-case comparison. The verification workshop at the end of the case study where I met the representatives from the other companies that participated provided a unique opportunity to see how we dealt with usability, compared to how this was done in other product development groups.” 1 At the time the recommendations were published ‘www.uselog.com | the product usability weblog’ had about 10.000 hits per months a significant portion of which originated from companies involved in product design and innovation (Van Kuijk, 2010)
- 38. 36 Design for Usability Methods Tools More information Card set The card set: Recommendations for Usability in Practice http://bit.ly/be-cardset rdset PhD thesis Managing Product Usability: How companies deal with usability in the development of electronic consumer products http://bit.ly/be-thesis Product usability weblog Weblog kept by Jasper van Kuijk during his PhD project www.uselog.com Response to recommendation #5: ‘Team: One roof: all disciplines - in one room - throughout the process’ Design researcher at strategic design consultancy “As a design researcher at [Strategic Design Consultancy], used to working in multidisciplinary teams, I very much agree with the statements you are making above. Both in my work with project teams and with clients, I have found that on top of this, there is an important distinction between ‘meeting’ and ‘working’ together. Even if being situated full time in the same space is not an option, there is a lot to gain by organizing work sessions when you get the chance, as opposed to meetings. The difference in my mind is that the latter is focused on sharing information, talking through issues and challenges; the former is focused on working through design challenges together, as a (multidisciplinary) team. In my experience this results in a mind-set that is much more positive, focused on solutions and understanding each other’s perspective, instead of underlining problems, disagreements and company politics.” Based on the input received, the recommendations were improved and a second round of feedback was collected. The recommendation card set was published together with the author’s PhD thesis, and can be ordered and downloaded through the Design for Usability website, and the author’s weblog.
- 39. 373.1 Organizational barriers and enablers for usability References [1] K. Malterud, “The art and science of clinical knowledge: evidence beyond measures and numbers,” The Lancet, vol. 358, pp. 397-400, 2001. [2] M. Steen, “The fragility of human-centred design,” PhD Thesis, Delft University of Technology, Delft, The Netherlands, 2008. [3] D. Wixon, “Evaluating usability methods - why the current literature fails the practitioner,” Interactions, pp. 28 - 34, 2003. [4] D. Norman, “Design as Practiced,” in Bringing Design to Software, T. Winograd, Ed., New York, USA: ACM Press, 1996. [5] R. K. Yin, Case study research: design and methods, 4th edition ed. Thousand Oaks, California, USA: SAGE publications, Inc., 2009. Author homepage Homepage of Jasper van Kuijk, highlighting education and research activities http://bit.ly/vankuijk Contact For more information about this research project, feel free to contact the author: Jasper van Kuijk Delft University of Technology Industrial Design Engineering +31 (0)15 278 1378 j.i.vankuijk@tudelft.nl Core publications Van Kuijk, J.I., H.C.C.M. Christiaans, H.Kanis, D.J. van Eijk (accepted with revisions) ‘Barriers and enablers for usability in practice – A multiple case study of product development groups of electronic consumer products’ Van Kuijk, Jasper, L. van Driel, D.J. van Eijk (accepted with revisions) Usability in practice; a cross-market comparison. A multiple case study exploring usability in product development practice in four markets’ Van Kuijk, J.I. (2010). Managing Product Usability; How companies deal with usability in the development of electronic consumer products. PhD thesis, Faculty of Industrial Design Engineering. Delft, Delft University of Technology.
- 40. 38 Design for Usability Methods Tools
- 41. 39 2 Result planning: When defining a user-centred plan of approach, a detailed insight into the desired product characteristics, intermediate process results and contextual conditions is required. In contrast to most methods, the UCD Kick-Off Tool helps developers to gain these insights. 3 Development method selection: Product developers tend to stick to product development methods they are familiar with without questioning whether these methods fit the intended development results. The UCD Kick-Off Tool helps developers to explore and select appropriate and feasible product development methods. 4 Method specification: The UCD Kick-Off Tool helps developers make a detailed description of all required development activities, guaranteeing that the intended development results will be achieved during the product development process. The UCD Kick-Off Tool is intended to be used by the core team1 of a development project during several workshop sessions at the early stage of a project, helping them to specify their plan of approach based on the design brief. 3.2 The UCD Kick-Off Tool A systematic support in defining a user-centred plan of approach Frederik Hoolhorst, Mascha van der Voort Summary If product development teams do not have the complete overview of how they will approach a product development project, or if an approach is not known across the team, this can be the cause of usability problems. Because in these cases executing essential user-centred design activities and integrating their outcomes are not planned for. The UCD Kick-Off Tool systematically supports product development teams to specify a detailed user-centred plan of approach. It is based on an extensive review of literature on design and product development methodology, as well as on observations and discussions in product development practice. The tool focuses on the following four areas (Figure 1): 1 Resolving stakeholder interests: Within most product development projects there are many stakeholders who have a wide variety of (sometimes conflicting or interrelating) interests that need to be met by the product-to-be-developed. The lack of an overview of these stakeholders and their interests is an important cause of use problems. The UCD Kick-Off Tool supports developers when making a complete, specified and prioritised overview of stakeholders for use in a user- centred plan of approach. 1 A core team consist of the leaders/managers of the departments or disciplines involved in a project.
- 42. 40 Design for Usability Methods Tools Authors Bios Frederik Hoolhorst is Postdoctoral Researcher at the University of Twente. He graduated as an industrial design engineer from Delft University of Technology (NL) in 2005. He then gained two years of experience as a product developer, before starting his PhD at the University of Twente in 2007. His research focuses on managing user-centred product development processes. Frederik’s PhD was supervised by Associate Professor Mascha van der Voort and Professor Fred van Houten. Mascha van der Voort is Associate Professor at the University of Twente (NL), where she heads a research group on Scenario Based Product Design and User-Product Interaction. Concurrently, she lectures on human factors, usability and research methods within the Industrial Design Engineering educational program at the University of Twente. Introduction In order to make the product development process as effective and efficient as possible, most companies have formalized how they conduct product development in their own proprietary product development methodology. However, these ‘corporate’ methodologies do not usually specify all the activities that need to performed and often do not specify the activities required to make the development process user-centred. There can be quite a gap between a corporate development methodology and a user-centred plan of approach. Unfortunately, product development teams often have an unspecific or incomplete overview of a user- centred plan of approach. For example, team members have different expectations regarding the desired characteristics of the product under development or regarding development activities that need to be executed. The lack of a detailed overview of the user-centred activities to be performed within a project is, based on the product-process relation, unlikely to lead to a product design that meets the intended use characteristics. There are a considerable number of methodologies for managing product development processes. Most address the importance of making a plan of approach, discuss what the most important issues are when making this plan of approach, and how to deal with the plan during product development. However, research into product development practice shows that user-centred plans of approach are mostly defined using team member experience, which in unfamiliar situations may cause teams to overlook important steps or aspects. There are no tools to support product development teams in defining a univocal, effective and complete user-centred plan of approach. Based on these insights, and through iterations of adjustment and evaluation with practitioners, a tool has been created that helps product development teams to specify a detailed user-centred plan of approach: the UCD kick-off tool. YouTube Watch a 2½ minute animated summary of this research: http://bit.ly/kot-summary UCD Kick-off Tool
- 43. 413.2 The UCD Kick-Off Tool Practice based research - The need for a product development support tool that is identified based on a literature study only does not necessarily have to be recognised by product development practice. Therefore two case studies were set up in collaboration with two companies. Both case studies provided additional insights into the expected circumstances under which the support tool would be used. Design support development - Based on the insights gained from the literature study and case studies, design criteria for the Kick-Off Tool were formulated. The development of Research method The development process of the UCD Kick-Off Tool consisted of four phases: Literature research - The development of the UCD Kick-Off Tool started with a literature study which provided insights into the state of the art of product development methodology. This study provided insights into the necessity of specifying product development methods in order to be able to create a detailed user-centred plan of approach. The insights resulted in a development statement and an overview of criteria regarding the to-be-developed support. Figure 1: Overview of the UCD Kick-Off Tool. With the design brief (left) as a starting point, the tool guides a team through the steps (middle) of 1) stakeholder mapping, 2) result planning, 3) method selection and 4) method specification to the desired end result: a user-centered plan of approach (right).
- 44. 42 Design for Usability Methods Tools Principles The UCD Kick-Off Tool is based on the following twelve principles: 1 Specify a dedicated user-centred plan of approach for each new product development project because the assignment and development context can vary greatly. 2 Ensure that each member of the product development team is familiar with the user-centred plan of approach as it provides insights into the exact output expected from the team members, which input can be expected by them, as well as the resources that can be used. 3 Make explicit expectations about use situations and usability-related requirements for the product under development. 4 Consider a development team as a dynamic entity: team members can be added or withdrawn from it based on skills and knowledge requirements. 5 Be aware of the project’s contextual agreements (such as procedures, , policies, rules or protocols) as well as the consequences of these for the product development process and product usability. 6 Identify all stakeholders which influence the product development process (including the development team members). For each of them, specify their interests as well as their possible contribution in the form of expertise/skills, decision taking, equipment, availability and budget. 7 State the expected end results of the project, as well as the intermediate results needed to get to these, both in terms of content (e.g., product proposition, requirements, interaction concept) and format (e.g., text document, sketch, interactive prototype). 8 Consider and investigate a wide range of methods and tools for reaching the intermediate results (i.e. beyond the range of methods and tools team members are most familiar with). 9 Base the selection of methods and tools on their the tool included a reflection step in which experts from the product development and design research fields were asked to reflect on the latest development state of the support tool. Their remarks and recommendations served as input for the next development iteration. This phase resulted in a method that helps developers to specify a user-centred plan of approach. Validation and evaluation - Developing a support tool that is directly applicable in product development practice was one of the main objectives of the product development support. The support tool was validated and evaluated in close cooperation with two companies. This process included several loops in which (1) the support tool’s viability was checked, (2) criteria regarding the format in which the support tool is made available were formulated and (3) the method was incorporated into its actual format. This phase resulted in a workshop manual that supports the execution of a series of workshops in which product development teams apply the methodology. By completing this series of workshops, product development teams can produce a detailed user-centred plan of approach. Result: The UCD Kick-Off Tool The research project resulted in the UCD Kick-Off Tool that meets twelve core principles specified below. The tool development was based on the literature-based development statement and an overview of criteria regarding the to-be-developed support formulated during the first phase of the research. The UCD Kick-Off Tool supports product development teams during the systematic definition of user-centred plans of approach based on a design brief for product development.
- 45. 433.2 The UCD Kick-Off Tool Workshop setup The workshop is divided into three parts, each consisting of several steps (see Figure 3): Part I - Stakeholder mapping Stakeholders are parties with an interest in the successful completion of a project, and thereby can influence the product development process as well as the resulting product. Therefore the goal of the first part of the workshop is to identify all stakeholders in the project (including the development team members). An overview is made of their interests, particularly regarding product use aspects, as well as their possible contribution to the project in the form of expertise/skills, decision taking, equipment, availability and budget. However, not all stakeholders’ interests are equally important and not all interests can be met. Therefore, the second step of the workshop focuses on prioritising stakeholders’ interests regarding the product and the project organization, based on their value in the product development project. Furthermore, dealing with a variety of stakeholders’ interests can be problematic or impossible during a product development process. Some stakeholders’ interests might be conflicting or even unfeasible. This first part of the workshop is concluded by identifying problematic situations so the team is aware of them and finds ways to possibly even avoid them. Part II - Result planning and development methods selection Because not all stakeholders’ interests are equally important and can be met, it is necessary for the team to be aware of those interests that should at least be met for the successful completion of the product development project. Therefore in part II of the workshop the first aim is to make a promising, workable combination of stakeholders’ interests that forms the minimum development requirements. effectiveness and efficiency to generate the required results as well as on their fit to the resources and skills available for this specific project. 10 Do not only specify which methods and tools will be applied, but also when, how, by whom and with what means. 11 Team members should be aware of which role(s) they have during the product development process, which deliverables are expected from them during each activity, in which form this should be delivered, and what the deliverables will be used for next. 12 Team members should have a good overview of the budget, planningand equipment per development activity. Manual and templates The UCD Kick-Off Tool can be used as the basis for a workshop. To assist product development teams prepare and execute the workshop templates were developed together with a workshop manual (see fig. 2). PDFs of the templates and manual can be downloaded from the DfU website (see ‘More Information’ at the end of this subchapter). Figure 2 – The manual for the UCD Kick-Off Tool workshop.
- 46. 44 Design for Usability Methods Tools To be able to define a user-centred plan of approach, product development teams need to have detailed insights in the desired product characteristics as well as the intermediate results of the development process. The following workshop steps support teams when making an elaborate product specification and when defining project milestones in terms of the product’s user group(s), use goals, context of use and other relevant product aspects. This is achieved by using the workable set of stakeholder interests developed in Part I of the process. Finally this part of the workshop looks at the selection of development methods and tools that will be used to create the desired deliverables. Selection is based on exploration of a wide range of methods. Part III - Development method specification The existence of a selection of methods does not guarantee that the desired deliverables will be produced and that the desired product characteristics will be achieved. This very much depends on how these methods are executed. Therefore specification of the actual application of the Define stakeholders 1 Prioritise interests 2 Verify interests 3 Select focusinterests 4 Define conditions 5 Makeelaborate product specification 6 Define milestones 7 Select developmentmethods 8 Specify developmentmethods 9 Communicate user-centred planofapproach 10 Designbrief Detailed user-centred planofapproach Workshop part I Stakeholder mapping Workshop part II Result planning and development methods selection Workshop part III Development method specification Figure 3 – Overview of the stages of the UCD Kick-Off Workshop selected product development method is needed. The third part of the workshop supports teams when making and communicating a concrete action plan for the product development process based on the product description, overview of milestones, overview of selected development methods and the overview of stakeholders. The action plan describes the required development activities, required input per activity, development techniques and the allocation of staff and resources (time, budget, equipment, etc.). Application domain Using the UCD Kick-Off Tool has added value in large- scale user-centred product development projects. In these situations, there are many project stakeholders whose interests need to be taken into account, and the knowledge regarding the required product’s main characteristics, especially regarding product use aspects, is limited at the start of the project. These projects often have a long lead time. The UCD Kick-Off Tool is preferably used at the start of a user-centred product development project, when the core
- 47. 453.2 The UCD Kick-Off Tool Evaluation and validation in practice When developing the UCD Kick-Off Tool’s methodological framework, the tool’s developers conducted observations in product development practice. To verify the framework’s comprehensibility and applicability in product development practice, it was then discussed with two companies: a midsize all-round design agency, and a large multinational producing printer-copier systems for the professional market. In these discussions the issue was raised how to design the UCD Kick-Off Tool in such a way that it could easily be applied to the product development practice. Based on this feedback a workshop around the UCD Kick-Off Tool was organised. Finally the comprehensibility of the workshop manual was discussed by three companies. The first two companies were midsize design agencies whose employees were experienced in the development of user-focused, mainly healthcare, products. The third industrial partner was a large multinational, producing a wide range of food products and products for personal care. team is setting up and planning the project. In later stages, the UCD Kick-Off Tool can be used by the core team to update and refine the user-centred plan of approach. However, using the UCD Kick-Off Tool can also be beneficial for smaller user-centred product development projects where it can be used as a reference to check whether all required elements of the user-centred plan of approach are defined and whether all the relevant aspects for defining a user-centred plan of approach have been considered. Benefits The UCD Kick-Off Tool is applied by the (core) product development teams during several workshop sessions. This ensures that not only the product development management team has an overview of the user-centred product development. Each team member has a univocal and complete overview of their user-centred product development process and the tool helps them makes explicit choices regarding the organization of this process. Many usability complaints are the result of important (conflicting) interests of stakeholders being overseen. The first step of the UCD Kick-Off tool explicitly supports the identification of all the stakeholders in product development projects (including the development team members). The tool is used to map their interests as well as their possible contribution to the product development process. Research shows that many product development practitioners tend to stick to product development methods they are familiar with without questioning if these methods fit the intended development results. The UCD Kick- Off Tool’s third step helps teams to explore and select appropriate and feasible development methods that will lead to the desired development results.
- 48. 46 Design for Usability Methods Tools More information Workshop brochure UCD Kick-Off Tool workshop manual and templates http://bit.ly/kot-brochure PhD Thesis Structuring user-centred product development processes http://bit.ly/kot-thesis Author homepage Frederik Hoolhorst http://bit.ly/hoolhorst Mascha van der Voort http://bit.ly/vandervoort The UCD Kick-off Tool Frederik Hoolhorst Mascha van der Voort Roland ten Klooster Partner in Plato Product Consultants and Professor Packaging Design and Management at the University of Twente On how the UCD Kick-Off Tool stimulates consideration of product use throughout the design process “Design teams spend attention to product use aspects rather late in the design process. Paying attention to usability already early in the design process enlarges the change to develop a product that meets the full range of use expectations of its users. This accounts especially for packaging, the field in which I am active. Changing a concept of a packaging design with thin walled constructions and make it user friendly is hardly possible when usability is not taken into account from the start. So, what I like about the UCD Kick-Off Tool is the fact that it stimulates design teams to consider product use aspects throughout the entire design process.”
- 49. 473.2 The UCD Kick-Off Tool References [1] J.I. van Kuijk. “Managing product usability: How companies deal with usability in the development of electronic products.” PhD thesis, Delft University of Technology, 2010. [2] P.H. den Ouden. “Development of a design analysis model for consumer complaints: Revealing a new class of quality failures” PhD Thesis, Eindhoven University of Technology, 2006. [3] J. Daalhuizen, P. Badke-Schaub, J. Fokker. “Community Based Design Support” I. Horváth Z. Rusak (Eds), in Proceedings of TMCE 2008 conference, Izmir, Turkey, 2008. [4] N. Cross. Designerly ways of knowing. Basel, Boston, Berlin: Birkhäuser, 2006. [5] N. Cross, H. Christiaans, K. Dorst. Analysing design activity. Chichester: John Wiley and Sons Ltd, 1996. [6] Kerzner, H. Project management: A systems approach to planning, scheduling and controlling. Berea, Ohio USA, John Wiley Sons, 2001. [7] PMI. A guide to the project management body of knowledge (PMBOK Guide). Newtown Square, Pennsylvania USA, PMI, 2001. [8] Dorst, K. “Design research: a revolution-waiting-to- happen.” Design studies 29: 4-11, 2008. Contact If you would like to know more about this research or would like to experience a workshop with the UCD Kick-off Tool first hand, feel free to contact the authors: Frederik Hoolhorst Industrial Design Engineering University of Twente +31 (0)53 4893 192 f.w.b.hoolhorst@ctw.utwente.nl Mascha van der Voort Industrial Design Engineering University of Twente +31 (0)53 489 2541 m.c.vandervoort@ctw.utwente.nl Core publications Hoolhorst, F. W. B. and M. C. van der Voort (2009). A concept for a usability focused design method. IASDR 2009. Seoul. Hoolhorst, F. W. B. (2010). Research project ‘Insight in design practice’: Insight into the user-centred design method’s applicability in design practice. Enschede, Universiteit Twente. Hoolhorst, F. W. B. (2012) Structuring user-centred product development processes, Engineering Technology, University of Twente, PhD thesis.
- 50. 48 Design for Usability Methods Tools
- 51. 49 methods is provided in a consistent, practitioner-oriented framework which supports correct execution. This has been implemented in a user interface that makes a growing collection of methods accessible to practitioners, researchers and students. The UCD toolbox has great potential: it can become a powerful tool for knowledge dissemination and it can ensure that (new) UCD methods, developed in the academic world, wil make it into product development practice. 3.3 UCDtoolbox.com Helping practitioners explore, select and apply UCD methods Tristan Weevers, Jasper van Kuijk Summary There are many development methods available which can be used for User-Centred Design projects. However, selecting one of these hundreds of methods can be a difficult and time-consuming task. This leads to many appropriate methods being left unused, although selecting the wrong method or using a method inappropriately can result in poor design decisions. Therefore, based on an interactive research and design approach, the UCD toolbox was developed that includes a new, online practitioner-centred UCD method selection tool. The development process included a literature study, interviews with practitioners, user evaluations and two conference workshops. This has resulted in a toolbox that includes system design, selection procedure, method presentation and user interface. The selection procedure, based on a set of criteria that matches the practitioners’ knowledge about the situation, reduces the time needed for searching for an appropriate method from days (in many cases) to less than an hour. Information about YouTube Watch a 3 minute animated summary of this research: http://bit.ly/tb-summary
- 52. 50 Design for Usability Methods Tools Introduction A crucial challenge within User-Centred Design (UCD) is getting users involved for example for obtaining information about their needs or evaluating concepts. Hundreds of UCD methods, tools and techniques have been developed with the goal of helping product developers with this challenge. These methods often serve specific needs such as evaluating rough, conceptual, designs of interfaces or learning about the behaviour and movement of users in environments. The product development practice can be extremely hectic and messy [1], leaving practitioners with very little time to explore and find a method or tool that matches their situation. UCD methods are spread over many different sources like scientific publications, books and online collections (see Results / Overview of existing UCD method collections). A number of attempts to collect these methods and tools have only resolved part of the issue: they provide an overview, but the number of methods is often limited and the resulting list or categories do not match a practitioner’s needs and preferences [2, 3]. Moreover, online method collections provide little or ineffective guidance for selecting the appropriate method [4, 5] and the descriptions of methods often lack the practical information needed to execute them [3]. Most information is given textually and without practical guidance, instructions, or examples [6], which (among others) can lead to a continued misuse of common methods such as focus groups and observational studies [7, 8]. The consequence of this inaccessibility and poor documentation of method descriptions, is that designers often stick to what they know, leaving many potentially beneficial methods unused [9, 10]. This hinders the development of the field and – eventually – the development of better products. It is also a barrier for the Authors Bios Tristan Weevers holds a Bachelor’s degree in Industrial Design from the Saxion University of Applied Sciences where he specialized in User-Centred Design. He then was awarded a Master’s in Design for Interaction at Delft University of Technology. His work was rewarded with presentations at the Intel Developers Forum (2009) and as a winner of the Design Challenge (2010). Tristan developed the UCD toolbox during his graduation project at TU Delft after which both parties agreed to turn the project into a startup. Tristan Weevers’ graduation project was supervised by Assistant Professor Jasper van Kuijk, PhD candidate Jaap Daalhuizen and Professor Daan van Eijk. Jasper van Kuijk is Assistant Professor at TU Delft with a focus on the practice of user-centred innovation. He obtained a Master’s degree in Industrial Design Engineering (IDE) at TU Delft, specializing in human- product interaction of electronic consumer products. He then worked two years as a design management consultant before starting his PhD in the DfU project.
- 53. 513.3 UCDtoolbox.com dissemination of new methods developed through academic research. Practitioners need to be able to quickly find the appropriate method, assess its qualities, and learn how to apply it. That is why, as part of the Design for Usability project, we started developing a new tool for selecting UCD-methods, one that is specifically tailored to the needs and preferences of UCD- practitioners. Research method During this project we focused on three issues: first, what is an appropriate procedure for selecting UCD-methods; second, how can these UCD-methods be presented so practitioners can compare and execute them, and third, how to design a user interface that facilitates these first two points. The design of the UCD toolbox was created through an iterative practitioner-centred research and design process, in which six phases can be distinguished (see figure 1). Results The outcome of this process was an overview of existing UCD method selection tools and their strengths and weaknesses, design criteria for a practitioner-centred UCD-method selection tool and the design for the UCD toolbox. Overview of existing UCD method collections To gain insights into the benefits and limitations of current method selection tools an analysis was made of these tools. Figure 1: The UCD toolbox development process
- 54. 52 Design for Usability Methods Tools categorizations are often not in line with the preferences, background knowledge and working conditions of practitioners. Finally most initiatives seem to ‘freeze’ once the project by which they were initiated stops [11]. During the user research and evaluations performed during the development of the UCD toolbox, it became apparent that very few of the existing method collections are known among UCD professionals. An overview is given below of six of these tools; the complete overview was made available as a card set (see ‘more information’ at the end of this subchapter). The method descriptions found in existing method collections are often incomplete and at times considered ‘too academic’ by practitioners. Most importantly, method collections provide little or ineffective guidance for selecting the appropriate method. Some offer a categorization of methods, but these UsabilityNet Methods Table UsabilityBOK Usability Planner Service Design Tools Generic Work Process AllaboutUX Description Co-created by Nigel Bevan in 2003 to ‘provide usability professionals with an authoritative website of resources’ Initiated in 2004 by the Usability Professionals Association to be a growing library on UCD content (not only methods) Initiated in 2010 by Nigel Bevan to support the selection of UCD methods to be applied in a project or organization Created in 2008 by Roberta Tassi (graduation student) further developed by two Italian universities in 2009 Created in 2009 by students and researchers from the Rotterdam University of Applied Sciences. Allaboutux.org (2010) provides information about user experience (UX) collected and maintained by volunteers Number of Methods 35 40 - 40 88 82 Selection mechanism No No Yes, 7 Filters No No Yes, 6 Filters Categorization Phase No No No Phase No Method description Text only: Description varies between methods. Contains at least a summary, benefits, planning, references next steps. Text only: basic description, (dis) advantages, how to, considerations and references. Text only: Short description with (if available) a link to the UsabilityBOK description Short description with a reference URL and case studies Text only: Short description and references to websites and articles for further reading. Text only: Short description with strengths, weaknesses, references and characteristics overview
- 55. 533.3 UCDtoolbox.com Requirements for a practitioner-oriented UCD method selection tool Based on the literature study and on user research and usage evaluations of concepts and prototypes with UCD- practitioners, the following requirements were identified for a UCD method selection tool’s 1) procedure, 2) information about methods and 3) user interface. “You want to decide which method you are going to do in about 10 minutes.” Team Manager Design Engineering of an industrial design agency Method selection procedure: 1 Practitioners should be able to find a suitable method when faced with an unfamiliar situation (e.g., research question, target group, resources) or when they simply want to ‘try out something new’, to keep developing their skills. 2 Practitioners need to be able to explore and compare new methods in one overview; 3 Selecting methods based on research goal and resources is the most practical approach; 4 Practitioners need to feel they have control over the system in order to trust the results: they need direct feedback on which ‘population’ of the methods are applicable given the selection filters used; 5 Experienced practitioners more often prefer a free order of input of selection criteria because they tend to be more explorative in their way of searching (going back and forth, adding, removing criteria); 6 Novices (in terms of UCD expertise) seem to feel more confident when sticking to a sequence of selection criteria provided by the system than experts; 7 Since a method can require a certain duration to execute it with a low workload (40 hours spread over a year) or a high workload over a short period of time (160 hours spread over a month), the selection criteria should include a distinction between timespan and man-hours; 8 Available budget is a popular selection criterion among UCD-practitioners, but because it is very hard to specify a method’s execution costs (as that depends on how it is executed), it is not possible to make a ‘clean’ selection based on budget; 9 Comments, reviews and ratings of methods by fellow practitioners are considered valuable information. “What I do like is to see other people’s experiences with products.” Creative Packaging Designer at a global fast moving consumer goods company Displaying information about methods 1 The information should be up-to-date; practitioners do not want to feel like they are relying on outdated information. 2 Practitioners stated that information of all the methods should be given in a consistent order and layout; this improves understanding and comparability; 3 Practical information about how to execute a method was considered more important than theoretical background and references, as is often provided by current collections; 4 The tool should give information about the execution of the method in such a manner that the user can execute the method appropriately. This also includes information on how to optimize the method for various situations; 5 The build-up of the method description should be ‘layered’, starting with a quickly scannable overview or front page (with information such as purpose and required resources) and providing more detailed information and examples on the ‘deeper’ levels; 6 The front page of a method description should list its purpose (the results it provides), advantages,
- 56. 54 Design for Usability Methods Tools Figure 2: The main screen of the UCD toolbox, with on the left the filter bar, on the right the collection of methods, and on top, the view settings. Interface design 1 To make the selection procedure and the method descriptions understandable and usable for both experts and ‘starters’, the interface should act as a guide, but all actions should be ‘free of choice’; 2 Explaining all filter options and other elements of the user interface is a critical issue, as incorrect interpretation of a selection criteria may result in the selection of inappropriate methods; disadvantages, in- and output. An introductory video and a bullet-list overview of method properties are also considered helpful information for getting an impression of a method; “As far as I’m concerned, you need something that immediately makes you aware of the difference between methods.” Product Designer at user-centered design agency
- 57. 553.3 UCDtoolbox.com The UCD toolbox design The requirements described above were implemented in the current design of the UCDtoolbox. This section describes the system as a whole, the method selection procedure it facilitates, the user interface design, and how the methods are described, in terms of text and visual arrangement . Primary components The primary components of the UCD toolbox are: 1 Database (library) of UCD method descriptions; 2 User interface optimized to guide and facilitate method selection; 3 Method selection procedure; 4 Standardized format for method descriptions; 5 Social layer, allowing users of the UCD toolbox to share and discuss methods, upload examples of how the method was used in different environments, and suggest additions improvements. Method selection procedure The selection procedure, the way in which a practitioner is guided to an appropriate method, is primarily based on a set of criteria that practitioners know by heart, that are easy to find out, or that are essential to selecting the appropriate method: Type of object that is being worked on (e.g., a physical product, interface or environment); Goal of applying the method: i) learning about users and their context, ii) synthesizing solutions, iii) simulating a design, iv) evaluating a design; Limiting factors and available resources (e.g., timespan and staff); Optional criteria (e.g., desired study location and participant details). In addition to a criteria-based search, methods can be explored by using a keyword search. The level of detail of the information presented about each method increases gradually as the number of remaining methods decreases, enabling comparison possibilities (Figure 3). By providing multiple ways of exploring and selecting methods and by increasing the detail of the method-information, users are encouraged to go through the methods in an explorative fashion and sample information about various methods. Method descriptions The method descriptions in the UCD toolbox (see Figure 3) consist of: Introduction with description, overview of how the method scores on the selection criteria, possible outcomes of executing the method, benefits, limitations and a slideshow; Examples of how the method was applied, provided by users of the platform; Detailed description with references to similar methods; Possibilities for customizing the method and step-by-step instructions; Downloadable templates and documents to support execution; Discussion area where people can ask for help and improve content. Current state and future plans Similar initiatives have often been frozen after they have been implemented, thereby only giving an overview of the methods available at the time the collection was created, and not reflecting any progress since then. For tools like ours to ‘stay alive’ they need to receive backing from people and organizations in the field, which is why we will support the tool with a community of users, contributors and reviewers. We will also add web 2.0 mechanisms to the website, such as ratings and reviews. This is also the reason why we are aiming to commercialise the UCDtoolbox, as this will result in more development power and content input. Currently we
- 58. 56 Design for Usability Methods Tools period will be open access. Benefits For practitioners Authoritative library: a large body of methods for User- Centred Design in one up-to-date location; are reviewing a number of business models. One of these is the so-called ‘Freemium’ model for adding content and functionality; basic method descriptions and functions will be freely available to users, whereas the more ‘expert’ content and functions will require a subscription. However, since an important part of the project was financed during the Design for Usability project, most of the results developed in that Figure 3: When a selection criterion is entered in the method filter bar on the left, all methods describing methods that are not applicable will disappear from the population (on the right) of remaining relevant methods. When there is space available, these method cards will increase in size and show additional information that is easily comparable.
- 59. 573.3 UCDtoolbox.com For the academic community Facilitating knowledge transfer (of UCD methods) from academia to practice; Using the tool for students’ study material; Faster dissemination: publishing time is usually less than a month, so there is little chance that knowledge is ‘outdated’; High impact: the ease of exploring new methods will ensure a quicker uptake of knowledge by practitioners. Validation in practice Development of the platform was performed through an iterative development process which included multiple evaluation moments with end-users (an overview of these activities can be found in the section on the research method). Fast filtering: practical filters to quickly find and compare the methods needed; Actionable information: obtain high quality detailed information written by domain experts; Supporting community: get in touch with professionals world-wide for help or to improve method descriptions. Figure 4: Overview of the four main categories of selection criteria:product type, research goal, resources, additional, and the options within each category (e.g., ‘product’ or ‘interface’). Figure 5: When a method description is selected, a detailed overview is presented, with an overview of properties of the method, a slideshow, examples, instructions, tweaks to optimize the method for various use situations, downloads and discussion boards.
- 60. 58 Design for Usability Methods Tools More information UCD Method Collection Card Set Overview of ten existing method collections http://bit.ly/tb-cardset UCDtoolbox website Stay informed about the progress on the UCD toolbox; access the trial version and contribute: www.ucdtoolbox.com lbox.com Master Thesis Tristan’s master thesis on the development of the UCD toolbox http://bit.ly/tb-thesis Sander Leer President of CHI Nederland, the Dutch chapter of CHI On the value of UCDtoolbox. com for the CHI community I first heard about the UCD toolbox when Tristan gave a workshop at the Chi-Sparks conference of 2011 that we, Chi Nederland, organized. From my own experience I know that our field is in great need of a thoroughly described and easily disclosed body of methods, and we believe the UCD toolbox has the potential of achieving this. Over the course of the years, a number of initiatives have been taken to create online method collections, but none has taken the practitioner-centred approach used for the development of the UCD toolbox. Therefore the selection procedure that the toolbox offers is well-attuned to how we believe practitioners prefer to select methods. The same is the case for how the methods are described. Finally, the UCD toolbox can facilitate the long sought after cooperation between academics and practitioners in the field of user-centred design. Thoroughly researched and evaluated methods are described by design researchers in such a way that practitioners value them, and the academic community receives feedback about their methods from these practitioners. As Chi Nederland we share the goal of facilitating the exchange of knowledge and experience in the field of HCI between academics and practitioners, and therefore we gladly support the development of the UCD toolbox.
- 61. 593.3 UCDtoolbox.com University webpage of Jasper van Kuijk More information about his research and educational activities http://bit.ly/vankuijk Contact If you would like to learn more about the UCD toolbox or if you would like to contribute by submitting methods or reviewing them, please visit www.ucdtoolbox.com or contact one of the authors: Tristan Weevers Delft University of Technology Industrial Design Engineering UCDtoolbox t.j.t.weevers@tudelft.nl / tristan@UCDtoolbox.com Jasper van Kuijk Delft University of Technology Industrial Design Engineering +31 (0)15 278 1378 j.i.vankuijk@tudelft.nl Core publications Weevers, T.J.T. (2011) Method Selection Tool for User- Centered Product Development. Delft University of Technology. Master thesis. References [1] J. I. van Kuijk, “Managing Product Usability; How companies deal with usability in the development of electronic consumer products,” PhD thesis, Delft University of Technology, Delft, 2010. [2] N. Bevan, “UsabilityNet Methods for User Centred Design,” in Proceedings of HCI International, Crete, Greece, 2003, pp. 434-438. [3] B. Tidball, Stappers, P.J., Mulder, I., “Models, Collections and Toolkits for Human Computer Interaction: What Can We Learn?” in Proceedings of the 24th BCS Conference on Human Computer Interaction, Dundee UK, 2010. [4] F. Ferre, Bevan, N., Escobar, T.A., “UCD Method Selection with Usability Planner,” in Proceedings of NordiCHI, Reykjavik, Iceland, 2010, pp. 829-830. [5] N. Bevan, “Criteria for selecting methods in user-centred design,” presented at the I-USED, Upssala, Sweden, 2009. [6] A. Seffah, Metzker, E., “The obstacles and myths of usability and software engineering,” in ACM 47 (Dec. 2004), 2004, pp. 71-76. [7] S. Greenberg, Buxton, B., “Usability Evaluation Considered Harmful (Some of the Time),” in Proceedings of CHI 2008 Florence, Italy, 2008, pp. 111-120. [8] J. Nielsen, “The use and misuse of focus groups.” IEEE Software [0740-7459], vol. 14, pp. 94-95, 1997. [9] J. Goodman-Deane, Langdon, P.M., Clarke, S. and Clarkson, P.J. , “Categorizing design methods: how designers view the roles of user methods in design,” in Contemporary Ergonomics 2008 (CE2008), Nottingham, UK, 2008. [10] C. Cardoso, Clarkson, P.J., Keates, S., “Can Users Be Excluded from an Inclusive Design Process?” in Proceedings of HCI, Las Vegas, 2005. [11] Battle, L., N. Bevan, et al. “About the Usability Body of Knowledge.” Retrieved March 26, 2012, 2012, from http://www.usabilitybok.org/about.
- 62. chapter 4 Anticipating usability issues 60 Design for Usability Methods Tools
- 63. There is a great incongruity in user-centred design, namely that activities that are often considered the core of UCD, user research and usage evaluation, are not the activities that make a product more usable. To make a product more usable, the information that is produced through user involvement should find its way into the design. In the end, the usability of a product is determined by the decisions that are made throughout the development process. The new methods and techniques presented in this chapter help product development teams to anticipate usability issues (both positive and negative aspects). By knowing what they know and do not know, teams can act and take measures to prevent problems from becoming reality, for example by setting up extra user research, or to build on expected positive qualities. 4.1 Preventing unawareness in usability related decision-making Why asking the right questions can be more important than getting the right answers Christelle Harkema’s work focuses on making product development teams more aware of the danger of being unaware of usability problems. Teams usually focus on the uncertainty of the information they do have about usability problems, but pay much less attention to the problems they might be completely overlooking: with potentially disastrous results. 4.2 Envisioning Use A workshop technique to share use-related knowledge in product development teams As part of the DfU project, Stella Boess, Mieke van der Bijl-Brouwer and Christelle Harkema collaborated on the Envisioning Use workshop. In this workshop design and development teams take stock of all the implicit and explicit knowledge that team members have about product usage. The teams then develop a shared frame of reference based on this knowledge. 4.3 Anticipating soft problems Using product properties and user characteristics to predict user (dis)satisfaction Cha Joong Kim investigated how certain combinations of product characteristics and properties of user groups can lead to soft problems (products not aligning with user expectations). This allows teams to, even before a project starts, be alerted of potential user (dis)satisfaction issues.
- 64. 62 Design for Usability Methods Tools
- 65. 63 acknowledgment that unawareness can influence usability- related decision-making, and 2) identifying and addressing sources of unawareness. 4.1 Preventing unawareness in usability related decision-making Why asking the right questions can be more important than getting the right answers Christelle Harkema Summary The source of usability problems can be traced back to the decisions made in the development process. This PhD research project aimed to improve the quality of these usability related decisions. Factors that influence the quality of usability related decision-making were explored through four different studies, of which three were executed in product development practice. The main contribution of this research is that it identified the influencing factor ‘unawareness’ on usability related decision-making, and subsequently explored which types of unawareness there are and what causes it. Shortly put, unawareness is ‘not knowing of not knowing’. By understanding the different types of unawareness and what causes them, product development teams will be able to prevent unexpected usability problems. Therefore a two-step approach is proposed for improving decision-making: 1) realisation and YouTube Watch a 2½ minute animated summary of this research: http://bit.ly/dm-summary Research Findings
- 66. 64 Design for Usability Methods Tools Author Bio Christelle Harkema obtained her BSc and MSc degree in Industrial Design Engineering at the Technical University of Delft. During her Master’s she focused on ergonomics and research. After graduating she worked for 2 years as a usability specialist for Indes, the Dutch design agency. Combining practice and theory, she did a PhD on usability at Eindhoven University of Technology. Christelle’s PhD was supervised by Assistant Professor Ilse Luyk, Professor Cees Dorst and Professor Arnout Brombacher. some unpleasant surprises during the final tests: users had difficulties understanding how to operate the product. Design adjustments and extra user tests were necessary to solve these issues. How could this happen? This is a real world example that took place at a leading multinational with seemingly all the knowledge and expertise in place to develop successful products. The multi-disciplinary team consisted of skilled and experienced members of various disciplines, including interaction designers and usability specialists. Many usability principles and techniques were available to them and still the users experienced usability issues with the product. The cause, in this case, was that, at the moment of decision- making, the team was unaware of the consequences of this decision to use certain buttons to operate the product. It has been shown that usability issues can be traced back to decisions made during development projects [2]. However, there are many other aspects that can influence decision- making: team dynamics, skills of individual team members, how knowledge on usability issues is collected, the stage of the development process; these can all play a role. Above all these, decision-making can be strongly influenced when an issue as intangible and fluid as usability is concerned. This research project was set up to improve usability related decision-making by identifying the critical factors that influence the quality of decisions in product development. Research method Four studies were executed to explore the factors that influence the quality of usability-related decision-making. To gain a better understanding of decision-making and factors that were known to influence it, we first conducted a review of the literature. One of the factors identified was uncertainty: ‘knowing that you do not know’. For example, you have doubts whether a user test identified all the Introduction A team in a room, exploring the possibilities for innovative technological solutions and creating design ideas for a new generation of a product. Its predecessor was launched only a few weeks ago, but its success made the demand for a quick follow-up necessary. The team is the same as on the first product and is focused on how to improve the first version. Unfortunately, what happens next is a series of large organizational changes within the company, resulting in serious changes to the composition of the team. This new team worked hard to implement the conceived technological improvements and executed usability tests to validate the improvements made to the user interface. Meeting time and budget resources were limited, as by now they were nearing the end of the project. Unfortunately there were
- 67. 654.1 Preventing unawareness in usability related decision-making usability issues or you are uncertain whether you picked the right user interface concept; you are not certain that you have either all the information and whether it is correct. The second study was an explorative study at a Dutch design agency. This study verified that uncertainty is an influencing factor on decision-making in design practice. However, it also raised a new question; what if a team is not uncertain but unaware. What if they are in a state of ‘not knowing that you do not know’. For example by not even considering that a particular design could lead to usability issues? In a third study, involving 14 interviews with core team members of a product development project at a leading multinational, it was confirmed that usability issues do result from unawareness during decision-making in the development process. The final and fourth study was an in-depth study to assess how unawareness leads to usability issues. The starting point for this study were usability issues that were identified in the product. Studying 2.056 project documents provided an overview of the process and the usability-related decisions taken during its course. This resulted in being able to distinguish between multiple types of unawareness and the identification of possible sources of unawareness. Results Decision-making Usability issues are a result of a mismatch between how the user uses a product (actual use) and the how the designer intended the use with the product (intended use) [1]. Intended use results from design decisions made during the product development process. Previous research has shown that ‘incorrect’ design decisions may result in usability issues [2]. Therefore the focus of this project was on usability related decision-making, (see Figure 1), and on finding ways to improve this decision-making, thereby reducing the number of usability issues. The development of electronic products is often executed by a team, operating in a certain ‘context’. This context has influential characteristics, such as time pressure, the presence of multiple stakeholders, and that the team has to deal with ‘ill-structured problems’. Time pressure in product development is generally high, as an early market introduction will give your company an advantage in sales and profit [3], while the product development financial resources are often limited. Secondly, today’s complex products cannot be designed by a single person, they require multidisciplinary teams, in which each stakeholder or discipline has its own viewpoint, knowledge and values from their own area of expertise [4]. The designer challenge is to balance and integrate the various perspectives to come to a good solution. A last example of characteristics of the context is that design problems are ‘ill-structured problems’, problems that are not fully defined [5], resulting in many possible solutions, each with its own advantages and disadvantages. Figure 1: The focus of this research project: decision-making in design practice
- 68. 66 Design for Usability Methods Tools absence of information, certain pieces of information can be missing, or the information can be unreliable. In addition to incomplete information, difficulties in understanding can make decisions harder to take. A good understanding of the information can be obstructed by ambiguous information, the novelty of situations, and fast-changing or unstable situations. Finally it can even be, there are conflicting alternatives, in which case the solutions have different advantages and disadvantages. To sum this up, uncertainty is about the certainty about answers to questions, while the following factor ‘unawareness’ is about asking the right questions. Unawareness During the exploratory phase of our research, we identified unawareness as an influencing factor on usability related decision-making. A subsequent literature study on the subject of unawareness showed that it is not a common topic in design literature. Therefore the majority of the studies Although there is no generally accepted model of decision- making in design, the following three phases can be distinguished [6]: 1 Situation recognition: based on team member expertise the situation is observed and assessed as being typical or novel, resulting in the choice of either a typical or a new, more challenging approach; 2 Serial option evaluation: evaluate the options that are generated and select a satisfactory one; 3 Simulation: imagine whether the selection option and action will result in a satisfactory solution; this can be achieved by mental simulation or by sketching or prototyping. 4 In this process, uncertainty and unawareness are influencing factors on usability related decision- making. To improve the quality of decisions, a better understanding of these factors is necessary. Uncertainty The explanation of the uncertainty factor is based on the results of the literature study and the explorative study in design practice. Our analysis of uncertainty, we discerned the following aspects; the types of uncertainty and sources of uncertainty [7] (Figure 2). The types of uncertainty describe what the decision maker is uncertain of: Uncertainty about outcomes Uncertainty about situation Uncertainty about alternatives Sources of uncertainty can be: Incomplete information Inadequate understanding Conflicting alternatives In our survey of the literature, incomplete information was the most often mentioned source of uncertainty. Information can be incomplete in varying degrees. There can be a complete DECISION-MAKING sources incomplete information inadequate understanding conflicting alternatives UNCERTAINTY uncertainty about outcomes uncertainty about situation uncertainty about alternatives types contributing influencing Figure 2: Types and sources of the influencing uncertainty
- 69. 674.1 Preventing unawareness in usability related decision-making Inadequate overview Fixation The source ‘inadequate consideration’ refers to the amount of attention paid to a topic. If usability is not on the agenda when taking a decision, the voice of the usability expert will not be heard, he/she will not be able to demand the required attention. The source ‘inadequate overview’ induces unawareness as a team does not have an overview ofall the variables involved , and therefore does not realise when information is missing. Fixation refers to holding on to a solution, finding it difficult to move away from a developed idea. For example fixation can occur when the usability expert on a team devotes all his/her energy into getting more information on this one issue, whilst ignoring other potential issues. These sources of unawareness can be related to question-asking: inadequate consideration would be ‘not asking questions’, inadequate overview leads to ‘not asking (all) the right questions’, and finally, fixation causes a team to ‘keep asking the same question’. These insights concerning uncertainty and unawareness will help to improve usability-related decision-making in design practice, which in turn can be expected to contribute to the level of usability of the products developed. If teams know about – are aware of – uncertainty and unawareness and what influences them, this will make them easier to recognise and address. For example, during the development of a product with a display and menu, uncertainty can occur about whether users understand the menu structure. By realising that this information is missing, an explicit action can be made. The uncertainty can for example be addressed by executing a usability test to find out more about users’ understanding of the menu structure. It is therefore relevant to explicitly state in this PhD research explored the topic of unawareness in decision-making. Unawareness is not an extreme type of uncertainty, it is a completely different concept: when a decision maker is unaware when assessing a situation, this can result in an incorrect assessment. For example; wrongfully thinking that all required information is available and proceeding with the decision-making process, may result in (unpleasant) surprises later on. As with uncertainty, we can distinguish between several dimensions of unawareness (see Figure 3): Types of unawareness: Unawareness about information Unawareness about consequences Unawareness about decisions Sources of unawareness: Inadequate consideration Figure 3: Types and sources of the influencing unawareness contributing sources fixation inadequate consideration inadequate overview unawareness about information unawareness about consequences unawareness about decisions UNAWARENESS types influencing DECISION-MAKING
- 70. 68 Design for Usability Methods Tools is overrepresented on the team, which tends to cause all the decisions to be made in favour of that discipline and other arguments to be put aside. We have shown that usability related decision-making is influenced by its context, uncertainty and unawareness. An overview of these factors is given in Figure 4. Workshop To disseminate the knowledge gained to product development practice in a way that suits practitioners, we developed a workshop titled ‘How to prevent unawareness in your design practice’. This workshop guides participants step by step through the theory by analysing one of their own projects. The usability problems of this product are listed, and then one of the problems is described in detail. This usability problem serves as a starting point for sketching a timeline of activities. The decisions that relate to the described the uncertainty and to verify whether the uncertainty has been addressed, before making subsequent decisions. Knowledge about unawareness during usability related decision-making can help the decision-maker to reduce the number of (unpleasant) surprises, i.e., unexpected usability issues. If team members know what causes unawareness and are on the lookout for these factors, this reduces the chances of usability-problems being caused by unawareness. For example, if team members know that when working on innovative products with complex technologies teams have a tendency to focus on the technological issues and pay much less attention to usability, this should alert the decision maker(s) to the potential danger of unawareness, and the realisation that they should widen their scope if they want to prevent usability problems. Another circumstance under which vigilance about unawareness is required is when one of the disciplines contributing sources fixation inadequate consideration inadequate overview unawareness about information unawareness about consequences unawareness about decisions UNAWARENESS types influencing DECISION-MAKING elements time pressure multiple players iterative design characterising DESIGN CONTEXT influencing sources incomplete information inadequate understanding conflicting alternatives UNCERTAINTY uncertainty about outcomes uncertainty about situation uncertainty about alternatives types contributing influencing Figure 4: Overview of influencing factors on usability related decision-making (other elements of the context are described in my thesis) Figure 5: Example of a result of the workshop, a timeline of activities and decisions made to identify influencing factors, in this example; unawareness.
- 71. 694.1 Preventing unawareness in usability related decision-making “Most designers have probably had the experience where you hit yourself on the head and say ‘Why didn’t I think of that?’ Why? Probably because we were focused on our own discipline, had limited time and budget, and had no idea that this certain issue would have critical impact. In other words: we were doing the best we could, but we were unaware. This workshop triggered me to be aware that I am unaware. Through thorough retrospective analysis of processes from the practice, my designer’s view was opened up and I got a broader perspective. Looking back at where critical issues were overlooked increased my awareness for future projects and decision processes. Because it’s easy to miss something you’re not looking for … (see ‘Awareness test’ on youtube)...” usability problem are added to this timeline. For each of these decisions, the group explores whether uncertainty or unawareness was involved, or whether the team chose to accept the presence of the usability issue and not deal with it. After each step, these examples are shared with the other participants, so that the participants are confronted with varying usability problems and development contexts. Benefits This research project took the concept of uncertainty from the literature on decision-making and specified how it applies to product development practice. In addition, we identified the concept of ‘unawareness’ and specified how it influences usability related decision-making product development practice. With explicit knowledge of these types and sources of unawareness, product development teams will be less likely to overlook usability problems. The first step is to realise and acknowledge that besides uncertainty, unawareness Figure 6: Evaluation of the insights from the research during the 2011 Design for Usability Symposium. Abbie Vanhoutte Usability designer at Océ Technologies On learning more about unawareness and usability problems influences usability decision-making in product development. The second step is to recognise and deal with the sources that lead to unawareness. Validation in practice The summaries of the results of the second explorative study were verified with each of the respondents. The results of the third study were verified with two key respondents and discussed with members the project team.
- 72. 70 Design for Usability Methods Tools More information Explanation and workshop guidance Workshop instructions and a poster that summarises and visualises the presented knowledge http://bit.ly/dm-brochure PhD Thesis Revealing unawareness in usability related decision-making http://bit.ly/dm-thesis Author homepage More information on the authors’ research and educational activities http://bit.ly/charkema During the 2011 Design for Usability symposium the workshop was held twice. It turned out that if the theory on uncertainty and unawareness could be applied to a familiar example, participants were able to quickly grasp the difference between uncertainty and unawareness. “Thank you for the workshop, you gave words to a very recognisable problem in product development” – Workshop participant The sources that define the factors uncertainty and unawareness, some of which were introduced spontaneously by the workshop participants, were discussed at the end of the workshop. The workshop provoked positive reactions from the participants about these newly learned insights; they considered the workshop both relevant and beneficial.
- 73. 714.1 Preventing unawareness in usability related decision-making References [1] S. Babbar, R. Behare and E. White, “Mapping Product Usability.” International Journal of Operations Production Management vol. 22(10), pp. 1071-1086, 2002. [2] P.H. den Ouden. “Development of a design analysis model for consumer complaints: Revealing a new class of quality failures” PhD Thesis, Eindhoven University of Technology, 2006. [3] A.C. Brombacher, P.C. Sander, P.J.M. Sonnemans and J.L. Rouvroye. “Managing Product Reliability in Business Processes ‘Under Pressure’” Reliability Engineering and System Safety, vol. 88, pp. 137-146, 2005. [4] B. Lawson and K. Dorst “Design expertise.” Oxford: Elsevier Ltd, 2009. [5] H.A. Simon. “The structure of ill-structured problems.” Artificial intelligence, vol. 4(3-4), pp. 181-201, 1973. [6] Klein, G.A. “A recognition-primed decision (RPD) model of rapid decision making.” in: Decision making in action: Models and methods. G.A. Klein, J. Orasanu, R. Calderwood and C.E. Zsambok, Ed. New Jersey: Ablex Publishing. 1993, pp.138-147. [7] R. Lipshitz, and O. Strauss. “Coping with Uncertainty: A Naturalistic Decision-Making Analysis.” Organizational Behaviour and Human Decision Processes 69(2), pp. 149-163, 1993. Contact If you would like to know more about this research project or would like to hold an ‘unwareness workshop’ at your organization, you are welcome to contact the author: Christelle Harkema Eindhoven University of Technology Industrial Design christelleharkema@gmail.com Core publications Harkema, C.L.E. (2012). Revealing unawareness in usability related decision-making. Eindhoven: Technische Universiteit eindhoven. PhD Thesis. Harkema, C.L.E., Luyk-de Visser, I.M., Dorst, K. and Brombacher, A.C. (2011) ‘Can existing usability techniques prevent tomorrow’s usability problems? In: Proceedings of International Conference on Engineering Design, Copenhagen.
- 74. 72 Design for Usability Methods Tools
- 75. 73 4.2 Envisioning Use A workshop technique to share use-related knowledge in product development teams Stella Boess, Mieke van der Bijl-Brouwer, Christelle Harkema Summary To be able to design products with a high level of usability, product development teams need to understand usability and users. For example, they need to know about the users’ abilities, and about future circumstances of use. However, our study of current product development processes has shown us that designers often have little direct contact with the actual use situation and end-users. When taking decisions, designers rely on their own implicit knowledge about product use and usability gained through personal experience, but they hardly ever share this process and knowledge with their teams. To stimulate the concept of usability in product development, we developed the ‘Envisioning Use’ workshop. It helps teams to establish shared goals for usability, as well as a sense of ownership for the usability of the future product. In this workshop, members of a product development team share their knowledge of and experience with the envisaged use situation in a number of informal ways. By the end of the workshop, the implicit knowledge – in team members’ heads – is made explicit in a shared frame of reference: the product use mind map. This can then be used to decide which items require action during the rest of the development process: which knowledge about users and use is uncertain or missing. Over the course of its development, the workshop has been applied numerous times in companies, with products ranging from business software to household appliances. To enable product developers to set up their own workshops, we have developed a workshop instruction booklet. YouTube Watch a 2½ minute animated summary of this research: http://bit.ly/eu-summary
- 76. 74 Design for Usability Methods Tools You’ve put a new product on the market, you’re proud of it, and reviewers and consumers start talking about it. They praise the style and the innovation, but soon enough complaints emerge about ease of handling and understandability. These examples of usability problems contribute to consumer dissatisfaction, to the extent that consumers may even return a product to the shop. Alternatively,, a high level of usability can enhance consumer satisfaction. However, it is not easy for product developers to anticipate future usability. Authors Bios Stella Boess is Assistant Professor at Industrial Design Engineering, TU Delft, specializing in qualitative usage evaluation methodology as an explorative and generative step in product and societal development. This includes techniques like rough modelling, interaction and experience prototyping, and collaborative analysis. Stella received her PhD from Staffordshire University in 2003, with a research and design project on designing bathrooms for older people. She works on freelance design projects alongside her academic activities. Mieke van der Bijl-Brouwer graduated cum laude in Industrial Design Engineering at TU Delft in 2002. She is now an Assistant Professor at the University of Twente and researches and lectures on subjects in the field of usability, user experience and scenario-based design. She was awarded her PhD for her research on ‘Design for Dynamic and Diverse Use Situations’ (cum laude) in September 2012 (see subchapter 5.1). The Envisioning Use technique is one of the means to achieve this. Christelle Harkema obtained her BSc and MSc degree in Industrial Design Engineering at the Technical University of Delft. During her Master’s she focused on ergonomics and research. After graduating, she worked as a usability specialist for Indes, the Dutch design agency, for 2 years. Combining practice and theory, she did a PhD on usability at Eindhoven University of Technology, which she completed in 2012. Introduction Did you ever buy a new digital camera that you thought you would thoroughly enjoy, only to find yourself thinking: “Why do I keep on accidentally activating the movie function? That’s so annoying. The display indication is impossible to see outdoors, and the touch screen activates all sorts of functions when I’m simply holding the camera. The designer must have had x-ray vision and tiny hands!” Does the following experience seem familiar to you as a product manager, product designer or usability specialist?
- 77. 754.2 Envisioning Use For example, when designing a digital camera, it is important to consider, but hard to predict, the users’ expertise and preferred level of menu complexity. It is also important to consider the influence of changing circumstances of use, such as weather conditions. These affect whether a screen is readable in bright sunlight, whether the buttons are controllable with gloves on when it is freezing, or whether the camera feels slippery with sweaty hands from the heat. Clearly, knowledge about the context of product use and about users is essential input into the product development process. Such knowledge could help when making design decisions that improve the usability of the resulting products, particularly when the knowledge is shared between team members. This study and the resulting workshop aim to support this sharing and application of knowledge. To gain insight into current product development practice and how it handles usability and knowledge about usability, we studied the product development process of design teams. The second goal of our research was to come up with improvements for the problems the design teams encountered. Based on the insights gained, the Envisioning Use workshop technique was developed, evaluated and refined. The technique serves to improve usability during the development process by providing teams with a shared vision on product use. Research method In order to gain insights into product development practice, and how designers deal with usability, we conducted fifteen in-depth interviews with industrial designers [1, 2], and developed three case studies for product development practice [3]. The interviews were conducted at the designers’ place of work, and during the conversation the designers used previously designed products as tangible examples. In the case studies, three design projects were studied retrospectively by means of a group interview and individual interviews with members of the design team [3]. Related research by Harkema [4] was the source of case studies on product development practice. The results of our research are summarized below. To overcome the limitations identified through the interviews and case studies, we developed the Envisoning Use workshop. The technique was developed iteratively over a period of two years, incorporating six iterations and three evaluations. Half-day workshops were held with companies participating in the DfU project, as well as with other companies. The evaluations were conducted using fictional design cases (a presentation microphone and a digital camera) and real design cases provided by the participating companies, ranging from business software to household appliances (Figure 1). Figure 1: The Envisioning Use workshop was developed through an iterative approach, alternating evaluations and adjustments of the technique.
- 78. 76 Design for Usability Methods Tools or not appropriate for this thing.” - Interaction designer While designers do share use-related knowledge amongst each other, there is not enough sharing of their implicit knowledge with other product development team members: “The designer mainly [analysed the user and the use context]. He did market research and made critical scenarios, but we didn’t make these very explicit in this project I think. I checked again, but really it was just in our heads.” - Product development team member Results (2): the Envisioning Use workshop To facilitate the exchange of implicit knowledge about product use and users, we developed the Envisioning Use workshop. Results (1): Interview and Case Study Research Our interview and case study research led us to conclude that product development teams do not have enough direct access to the context of use. They rely on their implicit knowledge and personal experience. They hardly ever share this knowledge because there are few opportunities for successful communication of this ‘informal’ type of knowledge. Implicit knowledge is difficult to address in formal design review meetings and procedures. A second consequence of the knowledge being implicit is that teams do not clearly distinguish between what they know and what they assume about product use. More knowledge sharing in product development teams would lead to better consideration of usability in design decisions, as has been concluded elsewhere [4, 5]. This sharing should happen informally and through a diversity of approaches that reveal different aspects of knowledge about use. We found that: Designers involved in product development often have little or no direct contact with the actual use situation and the end-users. Formal usability testing by experts, though well established, often confirms already known problems - too late for correction or too low in priority: “So when we tested the prototypes, we already knew that they were too heavy.” - Design manager of consumer electronics Designers often rely on their implicit knowledge and experience of usability when making decisions. They feel responsible and conduct informal user testing themselves: “And then I’d ask my girlfriend at home: ‘Do you like it or not?’ But in general I think it’s part of my job to decide what’s appropriate Figure 2: Through the steps of the Envisioning Use workshop, implicit knowledge of team members is made explicit in a product mind map
- 79. 774.2 Envisioning Use Approach and principles This half-day workshop supports the creation of a shared vision on product use by: Getting the members of a product development team to examine their design task from the user’s viewpoint rather than the developer’s viewpoint; Creating an informal environment in which team members can express and share their use-related knowledge and experience in several interactive ways. Use of this technique will not replace user testing; it can be seen as an ‘add-on’. Existing insights form useful input for the workshop, and the workshop can itself reveal a need for further user research. The basic principles of the workshop are: Eliciting real-life stories and envisaged scenarios [6, 7]; Making both facts and assumptions explicit: what participants know and don’t know about product use [4]; Figure 3: Selection of use phases on flip chart sheets Figure 4: Remembering experiences of use Structuring the information in a ‘product use mind map’. Workshop steps In the Envisioning Use workshop, team members access their personal knowledge and assumptions about product use in seven steps (Figure 2): 1 Remembering 2 Imagining 3 Structuring 4 Experiencing 5 Targeting 6 Envisioning 7 Questioning At the end of the workshop the available implicit knowledge about product use is made explicit in a shared frame of reference; the product use mind map, which can then be used to decide which actions are required during
- 80. 78 Design for Usability Methods Tools noted on post-its and attached to the flip charts (Figure 4). (For the ideas underlying the concept of use situations and use issues, please see (see the introduction of chapter 5.1). 2. Imagining To explore possible scenarios, participants now imagine situations in which the product could be used and what use issues might arise. As inspiration, associative materials (images of users and contexts) are provided (Figure 5). Participants might also now role-play different scenarios in a roughly simulated environment. Again the emerging use issues are added to the flip charts. ‘Assumptions about use’ have to be distinguished from ‘knowledge about use.’ 3. Structuring To create an overview, participants structure the information the remainder of the development process. The workshop also serves to enable team members to discuss use(r)- related topics more easily during the remaining process. 0. Preliminary step: setting up the product use mind map The team selects four to six use phases and the user roles to be explored in this workshop. Each use phase has a flip chart sheet forming the basis for the product use mind map (Figure 3). 1. Remembering To access the knowledge in the team about product use, participants tell personal stories about product use they either experienced themselves or have seen happening to others. These stories can also be based on observations made during user tests. The use-related issues in the stories are Figure 6: Participants working on the structure of the product use mind mapFigure 5: Participants imagining other possible use experiences through associative materials (pictures of users and contexts)
- 81. Figure 7: Participants mocking up quick prototypes and experiencing a scenario with a mock up of a digital camera. Figure 8: Targeting issues, for example by highlighting them with stickers in a product use mind map. 794.2 Envisioning Use on the flip charts and form categories (Figure 6). After clustering it is possible to go back to remembering and imagining phase and further complete the clusters. 4. Experiencing To explore use in yet more depth, participants now choose a use situation for a role-play. This can be based on the information on the flip charts, or a new use situation they want to learn more about. The role-play can be done with a simple mock-up of the product or a competing product (Figure 7). The use situation environment should be simply and roughly simulated. One or more team members are involved in the role- play, the others observe what is happening and add the emerging use issues to the flip chart. 5. Targeting The broad collection of information on product use has now to be prioritized so that it can be taken forward into the design process. This is achieved by means of targeting interesting and critical use situations and use issues (Figure 8). The situations define test conditions for later (user) tests, while the issues provide input into solution generation. 6. Envisioning To gather the product ideas that usually emerge in the previous steps, a brief idea-generation session can now be conducted. The participants quickly generate solutions for the chosen target issues, sketching them or creating quick, rough models (Figure 9). Positive and negative issues of the designs that emerge are added to the product use mind map.
- 82. Figure 10: Participants prioritizing use issues to work on in the further development process Figure 9: Participants showing a quick mock-up of an idea to other workshop participants in the envisioning step 80 Design for Usability Methods Tools documenting the Envisioning Use process and the product use mind map. Recommendations for executing the workshop The workshop can be set up by gathering the members of a product development team together for half a day either before or early in a development process. The participants should have knowledge of the product use of previous or related products and/or have influence on design decisions that influence product use. For example it is good to involve a usability engineer or marketing manager because of their broad user knowledge. It is also essential to involve a designer, project leader and/or engineer because they directly influence design decisions. Our experience is that a team of five is a workable number of participants. 7. Questioning In this step, participants reflect on all the knowledge that has been gathered in the product use mind map. Using post-it notes, they indicate knowledge gaps on the flip chart. The notes are then prioritized (Figure 10). The workshop concludes with the planning of further steps to address the most important issues. Wrap up All results of the workshop that have been gathered in the product use mind map, should be translated in a form that can be taken into the design process. This will differ per company or organization. Next steps should be planned to be able to answer the questions defined in questioning. These steps include planning actions to be taken, and
- 83. Figure 11: an example of a wrap up step: assigning actions to be taken. 814.2 Envisioning Use “You get an amazing amount of output, which is really helpful.” – Participant at design studio 1 Doing the workshop together keeps project team members dedicated to usability in the design process. “By experiencing this workshop, everyone who is working on the project stays dedicated and motivated.” – Participant at design studio 2 Knowledge can be shared in the team on topics that are familiar to experts, but unfamiliar to the others. “I am not part of the project, so I don’t know anything about [certain issue of the target market], but within four hours I do have an idea of what is playing a role there.” – Participant at software company Validation in practice Following the six pilots with fictive cases, three half-day Envisioning Use workshops were held with real cases in companies, two on household appliances and one on business software (Figure 1). After each edition the workshop was evaluated and adjusted accordingly. The workshops were recorded on video and the participants’ actions analysed from the video. Directly, after each workshop, the participants reflected on it in a group interview; later they filled in a questionnaire to assess the workshop’s effects. The workshop technique was found to be applicable to both projects aimed at redesigning an existing product and projects that have a more explorative character. With small adjustments it was found to be effective in all the cases in which it was applied. With products like household appliances, we conducted the workshop both Benefits In our initial research project, we identified a need for more sharing of use-related knowledge in product development processes. Individuals make small or large decisions based on knowledge and experience that remains implicit if it does not have a clear place or is not easy to back up. The simple and quick Envisioning Use technique enables teams to share knowledge, reflect on decisions and to consider usability earlier on and more easily in the development process. To be most effective, the first workshop should be conducted at or before the start of a design project. The workshop efficiently creates an overview of the use- related knowledge and knowledge gaps in the design of a product or service.
- 84. 82 Design for Usability Methods Tools More information The Envisioning Use workshop Mieke van der Bijl-Brouwer Stella Boess Christelle Harkema Workshop manual A booklet that helps you run Envisioning Use workshops yourself http://bit.ly/eu-booklet Author homepages More information on the authors’ research and educational activities: - Stella Boess http://bit.ly/boess - Mieke van der Bijl-Brouwer http://bit.ly/vanderbijl - Christelle Harkema http://bit.ly/charkema in contexts that mimicked real life environments and in a user test lab. When the topic of the workshop is software, it is recommended running more concise versions of the ‘experiencing’ and ‘envisioning’ steps, in order to be able to consider varying contexts. Stella van den Berg Manager Business Development Local Consumers Expert at Philips On how the Envisioning Use workshop fostered a joined feeling of ownership “Stella and Mieke facilitated the Envisioning Use Workshop for our project team when I was at Philips Consumer Lifestyle (Drachten, The Netherlands) in an early stage of the development of a household appliance. I think the three main benefits for our project team were: Starting from existing knowledge about product use. Sharing this knowledge within the team ensured that everyone started from the same background. Also, having a shared picture of the past made it easier to talk about the future. Getting out of our roles as engineers, designers and marketers, and getting under the skin of the end user. This was mainly done by experiencing different scenarios in a role-play. It made us look at the product from new angles and see details we would not have seen otherwise. Identifying attention points for usability aspects as a team. This created a joint feeling of responsibility to find solutions.”
- 85. 834.2 Envisioning Use Contact The researchers are available to run a workshop as an introduction to the technique. It can be customized to fit any design topic. Stella Boess Delft University of Technology Industrial Design Engineering +31 (0)15 278 3196 s.u.boess@tudelft.nl Mieke van der Bijl-Brouwer University of Twente Engineering Technology +31 (0)15 27 82551 m.vanderbijl-brouwer@ctw.utwente.nl Christelle Harkema Eindhoven University of Technology Industrial Design christelleharkema@gmail.com Core publications Van der Bijl-Brouwer, M., S. Boess and C. Harkema (2011). What do we know about product use? A technique to share use-related knowledge in design teams. Proceedings of IASDR 2011, the 4th World Conference on Design Research. Delft (NL). Van der Bijl-Brouwer, M. and S. Boess (2010). From remembering to envisioning product use: an informal design technique. Proceedings of the 7th International Design Emotion Conference 2010. Chicago (IL, USA). References [1] S. Boess. “Experiencing product use in product design.” in Proceedings of International conference on engineering design, ICED’09. Stanford University, Stanford, CA, USA, 2009. [2] S.U. Boess. “Meaning in product use: which terms do designers use in their work?” in Proceedings of DeSForM, Offenbach, Nov 6-7 2008, pp. 20-27, 2008. [3] M. van der Bijl - Brouwer and M. C. van der Voort. “Strategies to design for dynamic usability.” Proceedings of IASDR2009 Design Rigor Relevance. Seoul, Korea, Korea Society of Design Science, 2009. [4] C. Harkema, Subchapter 4.1 in this volume. [5] J.I. van Kuijk. “Managing Product Usability - How companies deal with usability in the development of electronic consumer products.” PhD thesis, Delft University of Technology, 2010. [6] T. Erickson. “Five Lenses: Toward a Toolkit for Interaction Design.” In: Theories and Practice in Interaction Design. S. Bagnara, G. Crampton Smith and G. Salvendy, Ed. Lawrence Erlbaum Associates, 2006, pp.301-310. [7] J.M. Carroll. Making use: scenario-based design of human-computer interactions. London: MIT Press, 2000.
- 86. 84 Design for Usability Methods Tools
- 87. 85 conducted four studies; three surveys and one experimental study. Our results suggest that soft problems vary along specific dimensions of user characteristics and product properties. These studies help companies to better understand their target users and products based on feedback from real product users. 4.3 Anticipating soft problems Using product properties and user characteristics to predict user (dis)satisfaction Cha Joong Kim “It works well according to technical specifications, but I would like to return it because I have a problem with…” Summary The consumer electronics industry is increasingly being confronted with consumer complaints which cannot be traced back to technical problems. This new class of consumer complaints is defined as ‘no-failure-found’ or ‘soft problems’. There are several possible explanations for this phenomenon: product development teams might not be taking these problems seriously, or the current range of products are becoming increasingly complex (product properties) and used by more and more diverse user groups (user characteristics). However, the definite causes of these complaints have not been identified. In order to reveal the interaction between user characteristics, product properties, and soft problems, we YouTube Watch a 2½ minute animated summary of this research: http://bit.ly/up-summary
- 88. 86 Design for Usability Methods Tools Introduction From the time that consumer electronics were launched on the market, consumers have complained about many of these products. Initially, these complaints focused on technical failure or product-malfunction, however over the years the numbers of these types of issues slowly decreased, as did the number of complaints. However, the late 90s witnessed an increase in consumer complaints, but this time the cause of the complaints did not seem to lie in technological failures [1]. According to recent studies, about half of the products that are returned are not actually broken. The products are suffering from what is known as ‘soft reliability problems’, a term introduced by Brombacher et Al. [2] to refer to the problems that cannot be traced back to a specification violation. So, the product is fully functional, does what it is intended to do, but yet users return it. Consumer dissatisfaction with regard to soft problems is presumably a consequence of a mismatch between user and designer/company expectations in product use [3]. Based on brand identity, use context, product properties and mediated by user characteristics (e.g., personality traits), users form expectations regarding the usability of a product [4]. However, the initial expectations that consumers have might differ from what they experience when actually using the product. Negative disconfirmation (underperforming in relation to expectations) leads to feelings of dissatisfaction (Figure 1). One of the reasons why the causes for the increase in complaints and product returns due to soft reliability are unknown is that it is common practice in the consumer electronics industry that customer complaints are dealt with by call centres. There are often very few direct links between these centres and the product development departments [5]. Figure 1: Conceptual framework of the study explaining how product and user characteristics interact and result in expectations about product use, which are then (dis)confirmed by actual product use. Author Bio Cha Joong Kim is working on his PhD at the Faculty of Industrial Design Engineering, Delft University of Technology in the Netherlands. He holds an MSc in Industrial Design Engineering from the same university. Since 2007 he has been involved in the Design for Usability project. His main research interests include user diversity, cultural differences in design, and cognitive aspects of human-product interaction. Cha Joong Kim’s PhD project was supervised by Associate Professor Henri Christiaans and Professor Daan van Eijk.
- 89. 874.3 Anticipating soft problems Despite this increase in soft problems, only a few studies have investigated which soft problems consumers encounter, let alone explain how user characteristics, product type and soft problems interact with each other. This project focused on improving the theoretical foundation and finding empirical evidence for soft problems. Research method Based on our collected knowledge, a design method was devised to help product development teams better understand the interaction between user, product and soft problems. First, we developed a conceptual framework based on the primary variables and the relations between them believed to play a role in the experience of soft problems (Figure 1). Next, for the evaluation of this framework, we conducted a total of four practical studies (three questionnaire-based surveys and a laboratory experiment). The first study identified which types of soft problems people experience. The second study explored mainly how user characteristics are related to specific types of soft problems. In order to study product type in relation to user characteristics more directly, we conducted a laboratory experiment. In this experiment participants were asked to interact with two products, an alarm clock and MP3-player, which were known to suffer from a low level of usability. The last study, again a questionnaire-based survey, was used to validate findings from the preceding studies and experiment. Results Types of soft problems faced by consumers. The first study mapped non-technical problems for a technically sound product which users experienced when interacting with household electronic products and services, and their reasons for complaining (or not). Interestingly, all the complaints are related to instrumental interaction, i.e. using, operating, and managing products [6]. Using product quality theory [7, 8], the types of problems found were categorized into three groups on the basis of the reasons why people were frustrated by their product(s) (Figure 2). Sensory quality This interaction quality is related to sensory perception. Consumers use their perceptive faculties to assess the structure, visibility, weight, sound, texture, and smell of a product. Judging this quality happens instantaneously while using, it is momentary, and based on human senses and can lead to either a pleasant or unpleasant experience. User dissatisfaction related to sensory quality is related to awkward product structure, visual hindrance, over- or low- weight, noise, irritating touch, and unpleasant smell. Functional quality This interaction quality refers to the extent to which users achieve their goals with the product. It is evaluated by assessing the results achieved after prolonged product use. Accordingly, the appreciation of this quality is not immediate but it’s effect is lasting. Complaints related to interaction quality mostly result from technological limitations or from a lack of product durability: for instance, functional constraints such as lack of function and incompatibility, low performance such as slow reaction and short battery life, unexpected errors, and frequent breakdowns. They are also related to poor product service. Operational quality Operational quality is related to cognitive and physical efforts that users have to invest to operate or interact with a product. The assessment of this interaction quality happens immediately and is long-lasting. Users evaluate ease of use, and the need for maintenance and repairs. Complaints related to operational quality surface if using the product requires continuous cognitive efforts or if users
- 90. 88 Design for Usability Methods Tools studies, these same products (alarm clock and MP3- player) were reported as being the most annoying product categories, but here functional problems played a role. This implies that there are differences between actual use and retrospective evaluation of soft problems experienced by users. For electronic products and related soft problems, see Figure 3. Relation between user characteristics and soft problems When research studies I in the field of product design look at user characteristics, they usually focus on demographic factors, such as age and gender, or on the difference between novice and experienced users. This is why we initially included as many user characteristics as possible, so as not to exclude potentially influential variables. Through the course of the three studies and the experiment, the most influential user characteristics were found to belong to three need to continuously check and take care of the product: for example, difficulty in understanding functions, confusing navigation, a nd inconvenient maintenance. This is often the result of a lack of information (feed-forward or feedback) or too many (redundant) functions. We used these three interaction qualities (sensory, functional and operational) to categorize soft problems in the subsequent studies. In these studies it became evident that participants mostly complained about poor performance of products, functional limitations (functional quality, about 41% of all the complaints), and difficulty in understanding functions (operational quality, about 34% of all the complaints). Complaints about sensory quality surface less often, but still form a substantial category (about 25% of all the complaints). In all questionnaire-based studies, the three types of soft problems showed the same pattern in terms of frequency. In the experiment, complaints related to functional quality were hardly reported. The most obvious reason for this difference was the fact that the two products used had no functional problems. Despite being hard to use, they did offer a sensible set of functions and worked correctly (technically). In the questionnaire-based Figure 2: Three categories of soft problems (sensory, functional and operational) and examples of manifestations of these interaction qualities. Figure 3: Frequencies of soft problems of alarm clock (upper) and MP3 player (below) between the questionnaire surveys and the experiment.
- 91. 894.3 Anticipating soft problems educated people regard operational quality as a major cause of dissatisfaction. This shows that educational level leads to different expectations of electronic product experience. Gender differences and household income were not found to influence the occurrence of (categories of) soft problems. Cultural background was also found to play a role in soft problems. Compared to Dutch and American respondents, South Koreans complain more about sensory and functional qualities of electronic products, and less about operational quality. Dutch respondents report the fewest sensory quality complaints and the most complaints related to operational quality, while American respondents report the fewest functional quality complaints. Although the scope of the study was limited to comparing these three countries, the findings provide a start for a better understanding of the influence of culture on the interaction with and experience of electronic consumer products. Cognitive aspects One of the cognitive aspects, low memorizing ability, was found to be related to complaints regarding operational quality. Prior experience with that particular product (expertise) is related to operational problems. People who have previous experience of an electronic product are more likely to complain about operational quality when using a similar product type again. Technical skill, use fixation (the state in which a user cannot find solutions while information provided by the product is contradictory or insufficient to guide the user to the proper operation), and familiarity with electronic products (a measure of prior experience not with specific products but with electronic products in general) did not influence the occurrence of soft problems. Personal traits Of all the user characteristics studied, only uncertainty categories: demographic factors, cognitive aspects, and personality traits (Figure 4). We conducted three questionnaire-based studies to identify significant variables in the interaction between user characteristics and soft problems, and at the same time validate these and filter out other variables. Our results show that the experience of soft problems is influenced primarily by the following (categories of) user characteristics: Demographic factors Young people turned out to be more sensitive to functional quality of consumer electronics than older people. The last group takes operational quality more serious as major dissatisfaction. Low-educated people take sensory quality more seriously when evaluating their electronic products, however well- Figure 4: The primary categories of user characteristics that surfaced in the study, and examples of user characteristics within those categories.
- 92. 90 Design for Usability Methods Tools avoidance was found to influence the occurrence of specific soft problems. People who score higher on uncertainty avoidance are more likely to complain about operational quality: people who are averse to unexpected events dislike unexpected errors or getting lost (e.g., in a menu structure). On the other hand, people who score lower on uncertainty avoidance are more likely to complain about functional and sensory qualities. It was also found that those people who complain in any situation, are most likely to complain about the sensory quality of their electronic products. Relation between product properties and soft problems In order to gain insights into the role of product properties, we investigated the following six dimensions in the questionnaire-based studies: operational transparency, physical interaction density, product importance, frequency of use, importance of usability, and perceived performance (Figure 5). The two uncommon dimensions, operational transparency and physical interaction density are explained in Figure 6 and Figure 7. The descriptions of the other four dimensions are more familiar or obvious. The studies show that low operational transparency is related to, as may be expected, more operational problems, while high operational transparency products are related to a higher number of sensory problems. High interaction density products also give rise to sensory problems. By contrast, low interaction density products are largely associated with operational problems. Comparing the influence of operational transparency and physical interaction, we found that operational transparency is a better predictor of the number of anticipated soft problems than physical interaction density. This implies that Figure 5: Product properties used to investigate influence on the occurrence of soft problems. Figure 6: Operational transparency indicates the degree to which interactive product behaviour can be deduced from its appearance or structure. Usually products with a high degree of integrated (information) technology like iPads or navigation devices, are less transparent than less high-tech products, such as washing machines and toasters. Figure 7: Physical interaction density refers to the frequency and duration of physical interaction between user and product.
- 93. 914.3 Anticipating soft problems related to complaints regarding the operational quality of the alarm clock (high operational transparency and low physical interaction density product). However, with the MP3-player (low operational transparency and high physical interaction density product) the complaints were closely related to problems with sensory quality. Furthermore, locus of control (the extent to which people believe they can control events that affect them) and cultural background are closely related to problems with products having high operational transparency and low physical interaction density, while prior use experience is related to soft problems with low operational transparency and high physical interaction density products. The impact of soft problems Soft problems do not necessarily result in product returns, but they do negatively influence the intent of future purchase. Follow-up (re)actions after having experienced problems are more likely. People will be more disloyal to the brand or seek redress directly through the helpdesk or the shop without waiting or staying calm. Particular soft problems lead to specific follow-up (re)actions: Functional problems were found to lead to negative comments about the brand, replacement by another brand, and/or calls to the helpdesk. On the other hand, people who face functional problems are least patient regarding having their problem solved in comparison to the other two problem categories. Operational problems often lead to taking follow-up (re) actions in any form. This is particularly true for simple products. Sensory problems in simple products do not always seem to lead to any follow-up (re)actions. People who are dissatisfied with the operational quality of complex products are less likely to take follow-up (re)actions than soft problems are more dependent on human cognition (information processing) than human perception (seeing, feeling, hearing). In the laboratory experiment in which people interacted with the MP3-player and the alarm clock, the type of soft problems experienced were related to the operational transparency and the physical interaction density of the product. For the alarm clock, an operationally transparent and low interaction density product, soft problems were mainly related to sensory quality. Complaints about the MP3 player, an operationally unclear and close interaction density product, were mainly related to operational quality. If electronic products are frequently used, this is likely to lead to a more frequent occurrence of soft problems. To be more specific, the more often a product is used, the more likely it is that these soft problems are sensory problems. It appears that there is a relationship between frequent use and increased exposure of our senses to the properties of the product. When people experience the performance of electronic products as being below their expectations, they are likely to complain more about the functional quality. This makes sense when one realizes that functional quality was considered to be part of the product performance category. On the other hand, people who consider the performance of their electronic product as being better than expected, pay more attention to sensory quality. Interaction between user characteristics, user characteristics and soft problems Certain user characteristics are related to specific types of soft problems, but the influence of the user characteristics is partly dependent on product properties. For instance, in our experimental setting, a ‘high proneness to complain’ is
- 94. 92 Design for Usability Methods Tools those who complain about sensory quality for the same product category. Interactive tool In order to translate our findings into a design language that can be used in design practice, we developed an interactive tool and a workshop. The tool provides practitioners with quick and easy information about the interaction between user, product and use problems (Figure 8). This kind of information is especially useful during desktop studies at the very beginning of a product development process. The workshop is a useful way of sharing a deep understanding and provides a hands-on experience on the interaction between user and product (Figure 9, 10 11). The workshop has the goal of making stakeholders in the product development process aware of the importance of soft problems, and to provide an in-depth understanding of target users for products in the development phase. Benefits Our study reveals that user characteristics and product properties play an important role in the occurrence of specific soft problems. A product development team can, at the beginning of a project, identify probable soft problems in terms of product properties and target group characteristics. For instance, if, as occurred in our study, a difference was found between people from different countries – a cultural aspect -, this could be a reason for a company to look for more knowledge about this subject by studying foreign target groups. Our findings are especially relevant when developing a new product, as these situations are often characterized by a lack of information. Expected soft problems can be identified in advance by defining the product in terms of product properties.Figure 10: Picture of card set for the workshop Figure 9: Post-it set for sensitizing session in the workshop Figure 8: Demonstration o f the interactive tool: an example page of characterizing a target user group in terms of user characteristics.
- 95. 934.3 Anticipating soft problems Figure 11: A brainstorming session at the DfU symposium Figure 12: A soft problem identification session at Océ
- 96. 94 Design for Usability Methods Tools This study gives an overview of how user characteristics and product properties interact with product use. If these aspects are taken into consideration during the product development process, product properties that lead to potential customer complaints can be identified early on, and the resulting product will increase consumer satisfaction. Validation in practice The Workshop method was validated through two workshops held at the Design for Usability Symposia (’09 ’11) and at Océ in the Netherlands, a global leader in digital document management and delivery technology. There were some differences between the two workshops in that participants at the symposium came from many different companies (Figure 11) while those at Océ were people from the product development team (Figure 12). As a consequence, products targeted in the workshops were different: in the symposium a broad range of electronic products were discussed, while printers and copiers were targeted in the Océ workshop (Figure 12). The symposium workshops were attended by 30 participants, while the workshop at Océ was held with 10 participants. Overall, participants liked the workshop structure and stated that the workshop inspired them, as our findings presented at the workshop provided a better and deeper understanding of how user characteristics and product properties interact in case of use problems. Our findings are particularly interesting because in current studies on actual use conducted by the companies, there were too few participants to evaluate the whole range of soft problems. “Most major company concerns in the product development process are mainly about operational quality because an electronic product is being armed with more and more functions. This workshop again confirms the insight that sensory and functional qualities are as important as operational quality.” Bert Ipema Senior Manager Product Research Centre at Philips Consumer Lifestyle On how this research changed his view on the importance of sensorial aspects “This workshop highlights the relation between product aspects and user characteristics. In that sense, it is a good addition to well-known usability methods such as personas and use scenarios. It enables designers to - during the development process - anticipate on specific usability problems that different target groups might experience.” Abbie Vanhoutte Usability Designer at Océ Technologies On the value of the workshop as a design tool
- 97. 954.3 Anticipating soft problems References [1] P.H. Den Ouden, “Development of a design analysis model for consumer complaints: revealing a new class of quality failures,” PhD thesis, Eindhoven University of Technology, Eindhoven, 2006. [2] A. C. Brombacher, P. C. Sander, P. J. M. Sonnemans, and J. L. Rouvroye, “Managing product reliability in business processes ‘under pressure’,” Reliability Engineering and System Safety, vol. 88, pp. 137–146, 2005. [3] E. den Ouden, L. Yuan, P. J. M. Sonnemans, and A. C. Brombacher, “Quality and reliability problems from a consumer’s perspective: an increasing problem overlooked by businesses?,” Quality and Reliability Engineering International, vol. 22, pp. 821-838, Nov 2006. [4] S. Donoghue and H. M. de Klerk, “Dissatisfied consumers’ complaint behavior concerning product failure of major electrical household appliances - a conceptual framework,” Journal of Family Ecology and Consumer Science, vol. 34, pp. 41-55, 2006. [5] A. Koca, “Soft Reliability in New Product Development: An Ontological Approach for Utilizing Field Feedback to Dynamically Sense and Adapt to Evolving Global Markets,” PhD Thesis, Eindhoven University of Technology Eindhoven, 2009. [6] P. Hekkert and P. Desmet, “Framework of Product Experience,” International Journal of Design, vol. 1, pp. 57-66, 2007. [7] D. Dantas, “The role of design to transform product attributes on perceived quality,” in Proceedings of the 6th International Congress on Design Research, Lisbon, 2011. [8] O. M. Madureira, “Product standardization and industrial competitiveness,” in Proceedings of the International Conference on Standardization and Quality, Sao Paulo, 1991, pp. 119-124. More information Card set Provides an overview of which user characteristics and product properties lead to which soft problems http://bit.ly/up-cards Contact If you are interested in learning more about this research, method or workshop, please contact the author. Cha Joong Kim Delft University of Technology Industrial Design Engineering C.Kim@tudelft.nl +31 (0)15 27 85146 Core publications Kim, C. Christiaans, H. (in press). User characteristics and behaviours in operating annoying electronic products. International Journal of Design. Kim, C., Christiaans, H. (2012). Soft problems with consumer electronics: the interaction between user characteristics and usability. Journal of Design Research, 10(3), 223-238. Kim, C., Christiaans, H. (2011). Usability problems: the influence of user diversity. IASDR 2011, Delft, the Netherlands. (selected as reviewers’ favourite)
- 98. chapter 5 Design Strategies for usability 96 Design for Usability Methods Tools
- 99. User tests do not make a product more usable, nor does user research: that is, if you do not apply this knowledge. This chapter outlines two new design strategies for making usable products. 5.1 Guidelines to design for dynamic and diverse use situations Exploring the who, where and why of product use Mieke van der Bijl-Brouwer developed guidelines for designing for dynamic and diverse use situations (DDUS). Dynamic use situations refer to the change of situations in time for one product, and diverse use situations to the change of situations in time and space for different versions of the same product. These new guidelines help designers to analyse DDUS, develop a ‘dynamic use mindset’, and develop solutions. 5.2 Anticipating soft problems Using product properties and user characteristics to predict user (dis)satisfaction Steven Dorrestijn thinks the other way around. Instead of asking the question, ‘how products should be adjusted to fit users?’, he investigated how products can be designed to deliberately influence user-behaviour and steer it in a (desirable) direction, and what the ethical implications are of doing so. To facilitate this innovative approach, he developed the Product Impact Tool and the Product Impact Workshop. 97
- 100. 98 Design for Usability Methods Tools
- 101. 99 5.1 Guidelines to design for dynamic and diverse use situations Exploring the who, where and why of product use Mieke van der Bijl-Brouwer American tourist Brian Wilson stood on a street corner in Amsterdam for 73 minutes and captured what to him was an amazing diversity of bicycle riders and use [1] Summary The aim of this study is to support product development teams in dealing with the variety of situations in which products are used, so-called dynamic and diverse use situations. Dealing with varying use situations in the design process is difficult because it is hard to predict the situations in which a product will be used, to anticipate what will happen when the product encounters those situations and to generate solutions for conflicting requirements. Our retrospective study of three design projects in practice showed that knowledge of dynamic and diverse use situations often remains implicit and is not shared between members of a product development team. This can have a negative effect on the validity of usability evaluations and can give rise to difficulties in decision-making with regard to product usability. We therefore developed a set of guidelines to support teams when dealing with dynamic use in the design process. The guidelines were developed iteratively and evaluated in seven student projects. They enable teams to create an explicit frame of reference of use situations which can be applied to contextualize usability evaluations; a ‘dynamic use mindset’ which inspires solution generation; and a shared vision on product use which supports decision making. YouTube Watch a 2½ minute animated summary of this research: http://bit.ly/ddus-summary Research Findings
- 102. 100 Design for Usability Methods Tools Design for DDUS is difficult because it is hard to: predict the variety of use situations a product will encounter: use situation analysis; anticipate what kind of issues will occur when the designed product interacts with these situations: use anticipation; deal with conflicting requirements from the different use situations in one design: solution generation. The relation between the different aspects of design for DDUS – use situation analysis, use anticipation and solution generation - is illustrated by the following example (See Figure 1). Designers of smart phones will undoubtedly do a great deal of research on who uses or would like to use smart phones, for which purposes, and under which circumstances. Apple designers might have expected that people would use the iPhone outside, also in cold weather (use situation analysis). However, that Korean people would use sausages to operate the phone in cold weather (use anticipation) was probably unforeseen! As the example demonstrates, each specific use situation can require different product characteristics. In this case, the problem can be solved by providing an accessory for the iPhone, such as a special glove (solution generation). Another difficulty of design for DDUS is that decisions need to be made on which use situations will be taken into account and which solution proposals will be chosen to suit these use situations. To make these decisions in a product development team, team members should have a shared understanding of use situations and related use issues. The importance of considering the variety of use situations is recognised in the main literature on usability. Many techniques are available to analyse use situations, such as interviews and observations [2], probing [3, 4] and after sales feedback [5]. Furthermore, the literature often mentions that the test conditions of usability evaluations should represent Introduction As opposed to tailored products, industrially manufactured products are used by varying users, for varying purposes in varying contexts. I have termed this dynamic and diverse use situations (DDUS). These situations refer to the change of situations over time for one product, for example, one day you might use your bicycle to quickly cycle to university to get to a lecture on time, while the next day you might use it to transport your groceries from the supermarket to your home. DDUS refer to the change of situations in time and space for different versions of the same product. For example, someone else might possess the same type of bike but only use it for recreational purposes, for example cycling with friends. Author bio Mieke van der Bijl-Brouwer graduated cum laude in Industrial Design Engineering at TU Delft in 2002. She is now an Assistant Professor at the University of Twente and researches and lectures on subjects in the field of usability, user experience and scenario-based design. She was awarded her PhD for her research on ‘Design for Dynamic and Diverse Use Situations’ (cum laude) in September 2012. She also contributed to the development of the Envisioning Use technique (see subchapter 4.2) Mieke’s PhD was supervised by Associate Professor Mascha van der Voort and Professor Fred van Houten.
- 103. 101 the actual user, goal and environment [6, 7]. However, in spite of these acknowledgements, little guidance is available on how an analysis and specification of intended use situations can lead to a frame of reference for usability evaluations. The purpose of our study therefore, was to develop a support tool aimed at filling this gap. Research method To analyse how designers in practice currently deal with DDUS, a retrospective study of three real-world design projects was conducted. Information about the projects was gathered by means of group and individual interviews with members of the development team of a product with varying use situations. Our analysis led to the problem definition, which we used as input for the development of a support tool which enables designers to deal with knowledge of DDUS in the design process. The tool consists of the Envisioning Use technique, which is discussed in Chapter 3.2, and a set of guidelines. The guidelines were developed iteratively in two educational projects, in which students designed for a real client. In the first project, four student teams designed a carrier bike for Bongo Innovations BV. The design processes of both projects were analysed by means of document analysis and a group interview. The guidelines were then revised based on these insights. In the second project, three student teams, with the help of the revised guidelines, redesigned an Airfryer for Philips. An evaluation of both projects led to a final workbook with guidelines. 5.1 Guidelines to design for dynamic and diverse use situations GOAL USER GOAL USER CONTEXT solution generation: creating solutions for conflicting requirements usability evaluation: evaluating and anticipating interactions and related usability issues use situation analysis: predicting and prioritising situations in which products will be used: users, goals and contexts use issues use issues use issue use issues use issues use situation: korean person in cold wearing gloves wants to use iPhone product: iPhone solution to accomodate this situation: special glove issue: people cannot control touch screen with gloves, instead they use a sausage Example: PRODUCT interaction interaction interaction interaction GOAL USER CONTEXT GOAL USER CONTEXT Figure 1: explanation of the different theoretical problems when designing for diverse use situations.
- 104. 102 Design for Usability Methods Tools Results The goal of the guidelines is to make designers more aware of DDUS, stimulate designers to analyse DDUS, explore the consequences of DDUS, apply the use situations consistently in the design process, and create a shared vision with regard to DDUS within product development teams. The guidelines are documented in a workbook, which also explains how they can be applied. A summary of the guidelines is included in this section. The design process of a hood for a carrier bike (see figure 2), by Industrial Design Engineering students at the University of Twente, is used to illustrate the application of the guidelines. Figure 2: the design of the carrier bike with a hood, which can be used in varying weather conditions to protect children. Figure 3: example of a part of the frame of reference for the design of the hood for a carrier bike. Relevant use situation aspects are indicated in pink and related use issues are indicated in yellow.
- 105. 103 The most important guidelines are: Make all members of a design team aware of dynamic use and create a common mindset by means of the Envisioning Use technique. Keep track of a consistent explicit frame of reference with use situations and related issues throughout the design process. Create this frame of reference by means of exploring use issues related to chosen use situations. Apply the frame of reference in usability evaluations. This ‘frame of reference’ is an overview of all relevant use situations that a product can possibly encounter and also lists the use issues such as usability or user experience issues that occur when a user and product interact in those specific circumstances. An example of a part of a frame of reference is shown in Figure 3. This shows how different weather conditions (use situations) relate to specific use issues for the design of a hood for a carrier bike. The shared vision on product use Creating a shared vision on product use means that all members of a product development team have the same mindset or ‘implicit frame of reference’ of relevant use situations and related use issues. These implicit frames of reference should be aligned with each other and with the explicit frame of reference (see figure 4). The best way of achieving this is to create an explicit frame of reference of product use together, as presented in the Envisioning Use technique (subchapter 4.2). The explicit frame of reference An explicit frame of reference includes two types of information: information about the diverse situations in which products are used, and information about the interactions between products and these use situations (see Figure 5). When a product is part of a specific use situation, this will result in an interaction with certain qualities: the use issues. Use situation aspects concern user characteristics, their goals and the context of use (see also Chapter 1.2). For the design of the carrier bike, examples are the physical characteristics of the cyclist, why the cyclist prefers a carrier bike to a car, types of luggage or passengers, road conditions, weather conditions etc. Use issues can be related to performance, usability or user experience. For example, if the box of the carrier bike is large enough to bring all preferred luggage (performance), if the hood is easy to adjust (usability) or if the children are happy to sit in the box (user experience). Besides this ‘use knowledge’, the frame of reference contains a target which defines which use situations and issues will be accounted for throughout the design process. 5.1 Guidelines to design for dynamic and diverse use situations Figure 4: The implicit frames of reference of product use (ideas about possible use situations and issues) of members of a product development team should be aligned with each other and with an explicit frame of reference, to create a shared vision on product use.
- 106. 104 Design for Usability Methods Tools Since the frame of reference can consist of a large collection of use situations and issues, it can easily contain too much information to deal with in communication or solution generation. We therefore propose the use of different views: a complete view which can be used as a check list in usability evaluations, and the simplified priority view which shows the most important situations and issues in one page, and which can be used to support communication and inspire solution generation. Figure 6 shows the design of the hood in a prioritized frame of reference of different weather conditions with explicitly mentioned use issues, represented in a storyboard format. Creating the frame of reference To create and update a new frame of reference, teams can employ different design activities. They can distinguish between internal activities, aimed at exploring how use situations relate to use issues based on assumptions, and external activities, aimed at exploring factual use situations and evaluating solution proposals in those use situations. Internal explorations include techniques like self-testing design proposals, scenario analyses, and the Envisioning Use technique. Internal explorations are important because they can easily be applied in an iterative design process: a solution can be created, explored quickly as to how it relates to different use situations, adjusted etc. Another benefit of internal explorations is that it guides the external activities by making gaps in factual knowledge or product use explicit. External explorations are activities aimed at gathering insights in the relevant use situations and issues for comparable products, for example by consulting online reviews or observations of use of comparable products. These explorations of current use lead to insights that can be extrapolated to future use. Finally, evaluations of design proposals in probable use situations can give insight in factual use issues. Since each Figure 5: relation between use situations, use issues and product characteristics and an example Figure 6: The design of the hood presented in the frame of reference of different weather conditions (use situations) and resulting experiences for parents and children (use issues).
- 107. 105 fit diverse use situations by stimulating the integration of use situations in the usability evaluations, which in turn can lead to use situation-specific recommendations for the creation of solutions. The activities mentioned in the previous sections are not meant to replace current design activities. On the contrary, they refer to activities that often implicitly – already occur in practice. The added value for design for DDUS is that the relation of these activities to the frame of reference with DDUS now becomes more apparent. The guidelines can therefore be applied in existing design approaches. Validation The guidelines have been evaluated in educational projects, as described in the research method. A validation in student projects enables close observation of the design process by multiple respondents. Their evaluations have led to the conclusion that working with an explicit frame of reference supports the generation of more focused research questions in usability evaluations, and offers opportunities for setting up more valid test conditions. A valuable approach to generating the explicit frame of reference is combining the exploration of the relation between use situations and usability with the verification of this relation. Moreover, the joint creation of this explicit frame of reference (such as within the Envisioning Use workshop) has led to a shared vision on product use in the design teams. Students highly valued this shared vision on product use, and found it beneficial in their team decision-making processes and in their creative process of solution generation. The workbook format of the guidelines did not always have the intended effects. Therefore future research will be aimed at developing other formats to improve the usability of the guidelines. Limitations Applying these guidelines will not lead to a company starting activity can be used to add, verify or remove information on product use, the frame of reference evolves in the course of the design process. Applying the frame of reference in usability evaluations The main advantage of a complete explicit frame of reference is its application in usability evaluations. Targeted use issues in the frame of reference can be translated into research questions for usability evaluations. For example, a question for the design of the hood for the carrier bike could be: ‘Can children communicate sufficiently with parents while seated in the box?’ The most important function of the frame of reference when planning usability evaluations is that it helps to set proper test conditions. To increase the ‘external validity’ of usability evaluations, the test conditions of these evaluations should reflect actual use situations as much as possible [8, page 241]. The frame of reference should give insights into what these actual use situations are. For example in the case of the carrier bike, apart from testing in different weather conditions, it makes sense to invite children of varying ages for a user test because their needs for communication with parents while seated in the box can be assumed to differ. Benefits The ultimate goal of the guidelines is to develop products with a high level of usability in the targeted diverse- use situations. This can be achieved by, on the one hand, creating design proposals with a high level of usability and, on the other hand, by better decision-making processes with regard to choosing the most appropriate solution and target use situations. The latter is achieved by stimulating the creation of a shared vision on product use. The former is achieved by firstly stimulating the designer’s awareness of dynamic and diverse use situations and thereby creating a mindset which can inspire solution generation. Secondly the guidelines support the creation of solutions that better 5.1 Guidelines to design for dynamic and diverse use situations
- 108. 106 Design for Usability Methods Tools to create usable products all of a sudden. The guidelines can only be applied successfully if two conditions in the company context (see also Chapter 2.1) are met, namely that usability is already considered an important issue by the product development team, and that the team is familiar with common usability methods such as usability evaluations. As mentioned, the framework has currently only been evaluated with students, most of whom have less design expertise than practitioners, and project circumstances differ from those in actual product development projects. These evaluations provided valuable insights, based on which the first iterations of the guidelines could be conducted. However, there are issues remaining to be explored which concern the application of the guidelines in product development practice, namely: Managing a frame of reference of use situations in combination with other references such as requirements and specifications; Creating a workable format for this frame of reference in design practice. Therefore we need to further validate the workbook in product development and design practice. Figure 7: Examples of the application of the guidelines to the design of a carrier bike: creating a first frame of reference in the Envisioning Use workshop, exploring use of a current solution, the product use mind map and a digitized version of the complete frame of reference.
- 109. 107 More information Guideline workbook This workbook describes and explains the guidelines to design for dynamic and diverse use situations http://bit.ly/ddus-guidelines The Envisioning Use workshop Mieke van der Bijl-Brouwer Stella Boess Christelle Harkema Workshop manual The Envisioning Use workshop manual describes how to set-up the workshop http://bit.ly/eu-booklet 5.1 Guidelines to design for dynamic and diverse use situations Pieta van der Molen Student Industrial Design Engineering On applying the guidelines for DDUS in a master assignment Industrial Design Engineering “The design for dynamic use guidelines were of great help during my research about the use and user experience of mobility scooters. A mobility scooter replaces walking, cycling and driving a car for people with impaired mobility and is therefore very diverse in use. The guidelines helped me to explore different aspects of the use of mobility scooters in different ways. By making these aspects explicit and thinking in use situations and use issues, it was easier to structure the large amount of information gathered. Moreover, it helped me to find and keep focus on what was important while designing new types of mobility scooters. With the help of storyboard scenarios, explaining the diverse use of the product ideas, possible future users were asked for their opinions. They immediately thought of themselves as driving the new devices. It resulted in very clear and specific feedback about how they might use and experience the new mobility scooters.”
- 110. 108 Design for Usability Methods Tools Core publications Van der Bijl-Brouwer, Mieke: Exploring Usability, design for dynamic and diverse use situations, University of Twente, PhD thesis, 2012 Van der Bijl-Brouwer, Mieke: Exploring design for dynamic use, proceedings of the international conference on engineering and product design education, 8 9 September 2011, City University, London, UK , 2011 Van der Bijl-Brouwer, Mieke and van der Voort, Mascha C.: Strategies to design for dynamic usability in Proceedings of IASDR2009 Design Rigor Relevance, Oct. 18-22, Seoul, Korea, 2009 References 1] B. Wilson. (2006, 10 September). Amsterdam Bicycles. Available: http://www.ski-epic.com/amsterdam_ bicycles/index.html [2] H. Sharp, Y. Rogers, and J. Preece, Interaction Design: Beyond Human-Computer Interaction, 2nd Edition ed. John Wiley Sons, Ltd, 2007. [3] B. Gaver, T. Dunne, and E. Pacenti, “Design: Cultural probes,” Interactions, vol. 6, pp. 21-29, 1999. [4] S. A. G. Wensveen, “Probing experiences,” in Proceedings of the first international conference on Design Emotion, Delft University of Technology, 1999. [5] J. I. van Kuijk, H. Kanis, H. H. C. M. Christiaans, and D. J. van Eijk, “Usability in Product Development Practice: After Sales Information as Feedback,” in Proceedings of IASDR07, Hong Kong Polytechnic University, 2007. [6] J. Rubin, Handbook of usability testing. New York: John Wiley Sons, 1994. [7] N. Bevan and M. Macleod, “Usability measurement in context,” Behaviour and information technology, vol. 13, pp. 132-145, 1994. [8] P. W. Jordan, B. Thomas, B. A. Weerdmeester, and I. L. McClelland, Eds., Usability evaluation in industry. Taylor Frances, 1996, p.^pp. Pages. PhD Thesis Exploring usability, design for dynamic and diverse use situations http://bit.ly/ddus-thesis Author homepage Homepage of Mieke van der Bijl-Brouwer, highlighting education and research activities http://bit.ly/vanderbijl Contact If you are interested in applying the guidelines in design practice, please contact the author for inquiries and support. Mieke van der Bijl-Brouwer University of Twente Engineering Technology +31 (0)15 27 82551 m.vanderbijl-brouwer@ctw.utwente.nl
- 111. 1095.1 Guidelines to design for dynamic and diverse use situations
- 112. 110 Design for Usability Methods Tools
- 113. 111 that products constrain users, and what does it mean when designers influence users in this way? The focus in this chapter is on the Product Impact Tool which was developed to make the research findings applicable in practice. It consists of a Product Impact Model and a format for executing a Product Impact Session. The tool aims at a change of mindset, to ‘think the other way around’: to not only consider how products serve user needs, but focus on how technologies guide and change users. 5.2 The Product Impact Tool Designing for user-guiding and user-changing Steven Dorrestijn Summary Research on human-product relationships often focuses on user research. Designers perform user research to identify user needs and characteristics, knowledge that enables them to design products that fit people. But products are not only adjusted to people, products also change people. To improve usability, we have to look at how products guide and change people. The aim of our study, was to investigate the impact of technology on users and how this knowledge can be applied in the design process. Can Product Impact knowledge help to anticipate and avoid use problems? Is it possible to design products that deliberately guide and change user behaviour? As part of our project we specifically considered the ethical dimensions of this view on technology and the design profession: is it acceptable The pitched roof of the trash bins at Dutch railway stations guides people towards its desired use: it prevents people from leaving rubbish on top of the bin. YouTube Watch a 3 minute animated summary of this research: http://bit.ly/pl-summary
- 114. 112 Design for Usability Methods Tools Introduction In order to design useful and user-friendly products, it is important to understand user needs and characteristics. However, it is equally important to see how technologies guide and change users. Clever use of these effects can improve usability and product acceptation whereas neglecting them often leads to product failure [1, 2]. A good case is the new electronic payment system introduced by Dutch public transport carriers (OV chip card). The system has been introduced nation-wide in the Netherlands by all the public transport carriers in all busses, Author bio Steven Dorrestijn followed a two-year program in Mechanical Engineering and Design History. He holds a Master’s in Philosophy of Science, Technology and Society from the University of Twente, and an additional Master’s in Philosophy from the University of Paris, Nanterre. His interest in product design and social aspects of technology were combined in his research on Product Impact, for which he was awarded a PhD from Twente University in October 2012. Steven’s PhD was supervised by Professor Peter-Paul Verbeek and Professor Hans Achterhuis. Figure 1: If the extraordinary curve in this bicycle lane in Paris makes you smile, this may also suddenly make you aware of to what a great extent our everyday movements are guided and constrained by technology.
- 115. 1135.2 The Product Impact Tool trams, the subway, and trains.. There have been many issues since its introduction some of which have been made news headlines, for example privacy issues concerning travellers’ data and hacking of the RFID chips by university researchers. When the public at large was introduced to the system in 2009, practical user problems also attracted a great deal of critical attention, especially the problem of forgetting to check out, a new and extra procedure compared to the old paper ticket system. The problem of forgetting to check out can be illustrated by my own experiences. Being curious about the OV chip card, I was happy to try it as soon as the system was first introduced in Rotterdam and Amsterdam in 2009. At first, the standard procedure for charging the card, and getting on and off a bus or tram seemed self-evident and easy. However, all the rest seemed to be quite difficult: extra subscription procedures for first use on the trains, very unclear installation of automatic money recharge, etcetera. Later on, I also found that the basic procedures for checking in and out caused major problems. After I had used the card a few times, I no longer felt confident about its use and was far from sure that I was using the card correctly. When I got a printout of my travel log at a machine, it appeared that I had made mistakes checking out and changing trams on all the four occasions that I had used the card. Every time people check in, a deposit is taken from the card. I had lost the 4 euros deposit on four occasions. My clumsiness was no exception. In September 2010 it appeared that the public transport companies had received half a billion Euros in deposit money as a result of these ‘incomplete transactions’ [5]. The case of the OV chip card is a clear example of a mismatch between the technical possibilities and characteristics and the consumer needs and practices. Partly, this mismatch stems from technical setbacks that can be overcome. However the OV chip card case also demonstrates the implications of technology on the way of travelling, and how the efforts required from consumers to learn new routines were completely underestimated. Technology should not simply be considered as a neutral means to fulfil needs that were always already there. Instead, technology can change users: it affects their behaviour, as well as their attitudes, needs, their vision on life, and way of living. The user-guiding and user-changing effects of technology form an important research topic in reflexive research fields – from philosophy and history to psychology [6, 7, 8]. To date, design practice has made little use of this knowledge, but there is a growing awareness of the possible advantages of combining these research fields [9, 10, 11]. This combination of both perspectives is innovative and promising for enhancing human-technology interaction and usability. This chapter presents the Product Impact Tool which was developed as part of the Product Impact research project. The OV chip card (Figure 2) case serves to illustrate how an analysis of behaviour guiding and changing effects of technology can contribute to increasing understanding and to reducing problems of usability and technology acceptation. Figure 2: Extra signage placed during the introduction of the OV chip card system.
- 116. 114 Design for Usability Methods Tools I will illustrate the model with modes of interaction and product impacts using examples from workshops in which an assessment and redesign of the OV chip card system was the subject (notably at the 2010 Design for Usability Symposium on Product Impact). Cognitive - Before the eye In the case of cognitive influence, technology is used to steer user behaviour through the user’s cognition. Cognitive interaction is about the perception and processing of information. In this quadrant of the model, products influence user behaviour through signs (e.g., arrows, texts, light signals, beeps). Behaviour-guiding through cognitive interaction means giving suggestions for use. Two variations of influence in this category are ‘guidance’ and ‘persuasion’. Examples Guidance: The OV chip card system could ‘guide’ travellers towards the correct procedures much more Research method An important part of the project was a literature review and a philosophical analysis of the different theories for understanding user-guiding and changing effects of technology. The ethical implications and challenges were extensively discussed. The connection between this theoretical approach and design practice was established by incorporating the knowledge in a design tool. The applicability of the tool was optimized during workshops with product development professionals. Result: Product Impact Tool The theoretical outcome of the project is an interdisciplinary, design and use-oriented perspective on the relationship in today’s society between humans and technology. Many papers have been published on different aspects of the project, for example, theories about product impact and usability, social engagement of designers throughout the history of design, and the assessment of ethical aspects of behaviour-changing technology. The Product Impact Tool was developed to make this perspective applicable to design and product development practice. The tool comprises a model (see Figure 2) that sums up and visualizes the basic ideas of how technologies guide and change users and a workshop session format (Figure 3) that provides concise advice on the application of the model. Both are explained below. Product Impact Model The Product Impact Model consists of a human figure, surrounded by four quadrants reflecting different modes of interaction: physical, cognitive, environment (indirect), and abstract (Figure 2). The model is based on the questions: what kinds of effects do technologies have on humans, and how do these effects reach the user? Figure 3: In the product impact model a human being (user) is represented receiving influences from different sides, through different modes of interaction.
- 117. 1155.2 The Product Impact Tool physical ‘coercion’ where the technology makes sure that travellers exercise the correct procedures for checking in and out. The design challenge of this type of system is to combine coercion with sufficient user- friendliness. Technically mediated gestural routines: Ultimately, the OV chip card and other components of the system should become part of the ‘user routines’. In routine- like behaviour, users have an intuitive relation with technology, so that they do not have to think about how to use this technology. At the moment, the system has been introduced but users are still learning to use it, users need extra help. Checking out with the OV chip card is not yet part of user routines, and apparently this is harder to achieve than the developers had assumed. Environment – Behind the back In the case of effects listed in the environment quadrant, it is technologies in the environment in which users reside that influence them. For example heating systems that increase comfort and allow us to live in cold regions. However undesired system effects can also be considered part of the environment: cars allow for fast transportation, but with too many cars, roads can get congested thereby limiting the possibility for fast transport. The environment we live in also shapes our moral standards: placing trashcans everywhere is a signal that throwing your trash on the ground is an undesirable action. Changing and designing the environmental setting is only possible to a limited degree. However, an exploration of the indirect effects of technology does help designers to grasp ‘trends’ that may converge or conflict. It can help them understand how experiences related to concepts such as privacy and freedom are co-constituted by the technical environment (‘environmental conditioning’). than it currently does. Though the massively employed pink colour coding attracts the attention of OV chip users to guide them to the check-in poles, the ,at times, illogical placement of these checkpoints has a negative influence, making people forget to check in and out. Better placement from the traveller’s perspective would help people not to forget. This design intervention could enormously reduce the numbers of check out ‘omissions’. Persuasion: The OV chip card system currently tries to persuade travellers (not just guiding actions, but teaching a lesson, adjusting people’s attitudes) by placing advertising campaigns and messages from the speakers in trains and busses. Persuasion could however also be attempted by more direct use of interaction with the system. In the workshop, we considered how the card and gates themselves could persuade travellers to check in and out by making the interaction more challenging: for example, introducing a game element, ‘every tenth passenger travels free’, was one of the ideas. Physical - To the hand The most obvious influence that technologies have on humans is that of physical behaviour steering effects. A characteristic of this influence is that the decision making process is largely cut short. Influences by physical interaction are obvious and widely applied in the form of technical obstructions such as fences, locks, et cetera. This interference in a user’s bodily gestures seems to be perceived as being more intrusive than product impacts that address a user’s cognition. Two effects in this category are physical ‘coercion’ and ‘technically mediated gestural routines’. Examples Coercion: The gates for the OV chip card system, applied in many subway stations, are obvious examples of
- 118. 116 Design for Usability Methods Tools important, because generalizing conceptions about the interdependencies between humans and technology marks people’s visions on technology. Discussions and controversies about technology often get bogged down in the extreme positions of ‘utopian technology’ and ‘dystopian technology’. Example In the case of the OV chip card this is very clear regarding the privacy issue. The security and privacy debate induced by the card hackers constantly alludes to the fear of a ‘definitive demise of privacy’ and the need for an ‘absolutely secure chip’. This idea that technology can be completely secure and controllable is a ‘utopian’ conception of technology. Its counterpart, the belief that the OV chip card system is the next big step toward Big Brother, is a ‘dystopian view’. The Product Impact Session In addition to the model, the tool contains a format with directions for conducting a Product Impact Session which helps to apply the product impact model to discover user-guiding and user-changing effects of a product, and generate ideas for redesign. Step 1: Preparation The first step of a product impact session is to answer some preparatory questions in order to focus on the specific design challenge. There are many aspects of a design assignment, so focus is a necessary and unavoidable step. This step helps identify critical behaviours, for example in the case of the OV chip card, the problem of ‘forgetting to check out’ is a critical use procedure. Step 2: Assess and re-design In the central phase of the Product Impact Session a Example An analysis of the routines of travel in relation to the technical environment reveals different trends that the public transport chip card interferes with. The OV chip card promises ease of use: fast and easy check in and check out, jumping on and off trains, switching between train and subway, etcetera, while payment proceeds automatically. This flexibility indeed matches a ‘trend’ of our time, conditioned by all kinds of network technologies in our ‘environment’: we have permanent access to the Internet for the weather forecast, banking, e-mailing etcetera. As soon as people become used to the e-payment card, the activity structure of pre-planning a trip for the whole day, buying a ticket, and then sticking to the plan for the day, will very soon begin to feel outdated. The old paper ticket was, as much as the new chip card now is, part of a regime that structures our behaviour, and that ‘conditions’ particular experiences of freedom and privacy. Nowadays, freedom is increasingly associated with flexibility. Abstract – Above the head The three preceding interaction modes, physical, cognitive and environment are about concrete relation between humans and technologies. This means that there are always concrete cases and examples at the base of the analysis. In contrast, an abstract approach results in generalizing theories and claims about the relations between humans and technologies. What is the nature, or the essence, of technology? Can we determine the course of technological developments, or does technology determine the course of human history? Obviously it is not in the power of designers, nor of users, to change how technology influences humans throughout history, on a global scale. Still, this abstract dimension is
- 119. 1175.2 The Product Impact Tool or prototype (combined and integrated with user tests). In all cases,, what is important, is to adapt the product impact mindset of seeing what the actual behaviour effects of a product are, irrespective of the (doubtlessly good) intentions of the designer. Step 3: Wrap-up The final step of the Product Impact Session is to simply wrap up the findings and ideas. A product impact brainstorm session helps to assess and redesign user-guiding and user-changing effects that are behind many of these use problems. The abstract category of ‘effects of technology’ gives the best understanding of the debate; but for re- evaluating and improving everyday practices of user- technology interaction, the concrete quadrants of product impact analysis are crucial. Benefits A clearly proven benefit is the relevance of the Product Impact Tool for educational purposes. Students as well as design practitioners appreciated learning the perspective of ‘thinking the other way around’; focussing on products to consider the user-changing effects. A broader societal benefit is that the perspective of Product Impact gives new impetus to the social engagement and responsibility of designers. Product Impact research provides insights and tools to revive the social role of design on a moderate but much more concrete scale . Validation in practice The Product Impact Tool in its current form is the result of workshops with companies involved in the Design for Usability project, with participants at Design for Usability symposia and students in design classes. These sessions have always proved to be inspiring for participants. product is assessed to identify user-guiding effects, at the same time prompting ideas for re-design. A session can be carried out at every stage of product development, although the objectives will differ. In the early stages of product development, the tool can support the definition of use scenarios. In the final stages or in the case of redesign, it can help to identify use problems with the actual product Product Impact Session Explanation In a Product Impact Session, a product is analysed with the purpose of discovering and designing user-changing effects. Results Wrap up Identified effects Design alternatives Preparatory questions Is the product necessarily encountered so that it can enforce behaviour? Or, is it a consumer product that can be easily avoided, and can rather only seduce users? Are there specific behaviour goals: usability, energy-saving, social empowerment? What are critical use actions that must be avoided or assured? Assess and re-design Mind set: Think the other way around! - Do not go from user needs to technical solutions, but from a product (or concept, prototype) to user guiding and changing effects. Use the model - Make a round along the quadrants of the model. - Do the interaction modes apply, and what effects can be identified? - Consider design alternatives to better guide users. - Try changing between cognitive and physical interaction. - Try to improve connection to trends in the technical environment. Figure 4: The steps of the Product Impact Session, a workshop conceived to provide product development teams with a reverse perspective on human- product interaction, namely not on how to design a product to fit people, but how a product could and might change people.
- 120. 118 Design for Usability Methods Tools More information Product Impact Tool Web-based repertoire of examples http://bit.ly/pi-tool In order to design good, useful, user-friendly products, it is important to understand user needs and characteristics. But, equally important is to see how technologies shape and transform user needs and behavioural routines. To improve usability, the focus must not just be on user needs, but also on how products guide and change people. Examples of user guiding design The pitched roof of the trash bins at Dutch railway stations prevents people from leaving rubbish on top of the bin, and guides them towards desired use (the cup on the roof in the picture is a trick). If this extraordinary curve in this bicycle lane in Paris makes you smile, it may also suddenly make you aware of the great extent our everyday movements are guided and constrained by technology. The usability of this remote control is awful. Users were even confused about which side was the front. The sticker (taken from a piece of fruit) at least solves this problem by guiding users when picking it up. The reconfiguration of behavioural routines and preferences by technology is an important topic in the philosophy of technology. To date, design practice has made little use of this knowledge. The Product Impact project therefore investigates how knowledge of behaviour changing effects of technology can be integrated in product design. Can Product Impact knowledge help to anticipate and avoid use problems? Is it possible to design products that deliberately guide and change user behaviour? It is also explicitly part of the project to consider the ethical dimensions of this view on technology and the profession of design. Goals The Product Impact project aims to improve understanding of how users change in the process of interaction with products, and to integrate this knowledge in design practice, by means of a Product Impact Tool. In this way the study contributes to improved understanding of human-technology interaction and the practice of design for usability. Results The Product Impact research has resulted in papers and publications on relevant theories about product impact and usability, on changing humans and society in the history of design, and about analysing the ethical aspects of behaviour changing technology. The Product Impact Tool, translates this research to practice by offering of a model for framing different types of product impact, and a format for organising a Product Impact Session. Innovation The Product Impact project combines knowledge from philosophy and behavioural sciences with engineering and design in an innovative way. In engineering technology is usually considered as an instrumental means to fulfil human needs. In philosophy and social sciences technology is often shown to change people in ways they had not foreseen themselves. Therefore, technology changes humans, and should not simply be considered as a means to fulfil needs that were already there. The recombination of both perspectives is innovative and promising for enhancing human-technology interaction and usability. Project For more information, examples, publications, and contact details check out ‘product impact’ on www.designforusability.org Product Impact User guiding and changing design Steven Dorrestijn Design for usability IOP-IPCR Product Impact Tool brochure More information on the Product Impact model and description of the workshop http://bit.ly/pi-brochure PhD Thesis Extensive theoretical background with references, explanation and discussion http://bit.ly/pi-thesis Onno van der Veen Owner/director at Zeeno Human-Centered Design On the Product Impact Tool and grand challenges Willem Mees van der Bijl Account and project manager at Indes On getting a fresh perspective through the Product Impact Tool “Design is progressively concerned with finding solutions for grand challenges. Energy consumption and sustainability, or supporting elderly people to live on their own, are examples. Stimulating desirable behaviours is an important aspect. The Product Impact Tool offers an interesting perspective for designing for behaviour adaptations.” “The Product Impact Tool offers a fresh perspective on existing products as well as products in development. A session can lead to interesting reflections on a product, but also result in surprising new product ideas: iInnovations that are distinctive; big leap innovations. In this way it contributes to the discovery of new market opportunities.”
- 121. 1195.2 The Product Impact Tool References [1] D.A. Norman. The psychology of everyday things. New York: Basic Books, 1988. [2] P.-P. Verbeek and A. Slob. User behavior and technology development: Shaping sustainable relations between consumers and technologies. Dordrecht: Springer, 2006. [3] P. van den Broek. “De schokgolf na de ontmanteling.” Vox 8(15), pp. 14-18, 2008. [4] J. Koot, “Reizigers verliezen iedere maand half miljoen euro door chipkaart,” Financieel Dagblad, 24 September, 2010. [5] P.-P. Verbeek, What things do: Philosophical reflections on technology, agency, and design. Pennsyvania: Pennsyvania State University Press, 2005. [6] M. McLuhan. Understanding media: The extensions of man (Critical edition by W. T Gordon). Corte Madera, CA: Gingko Press, 2003 [7] E. Tenner. Our own devices: The past and future of body technology. New York: Alfred A. Knopf, 2003. [8] N. Tromp, P. Hekkert P-P. Verbeek, “Design for Socially Responsible Behavior: A Classification of Influence Based on Intended User Experience.” Design Issues 27(3), pp. 3-19, 2011. [9] R.H. Thaler C.R. Sunstein. Nudge: Improving decisions about health, wealth, and happiness. New Haven: Yale University Press, 2008. [10] B.J. Fogg, B. J. Persuasive technology: Using computers to change what we think and do. Amsterdam; Boston: Morgan Kaufmann Publishers, 2003. Author homepage More information on the author’s research and educational activities: http://bit.ly/sdorrestijn Contact If you are interested in learning more about the Product Impact Tool or would like to hold a Product Impact session at your organization, please contact the author. Steven Dorrestijn Twente University Department of Philosophy +31 (0)53 4894811 s.dorrestijn@utwente.nl Core publications Dorrestijn S. (2012). The design of our own lives: Technical mediation and subjectivation after Foucault. PhD Thesis. Twente University, Enschede, The Netherlands. Dorrestijn, S. (2012). ‘Technical Mediation and Subjectivation: Tracing and Extending Foucault’s Philosophy of Technology’. In: Philosophy Technology, 25 (2), pp. 221-241. Dorrestijn, S. (2011) ‘Gedragsbeïnvloedende techniek en usability’. In: Tijdschrift voor ergonomie36/1, pp. 5-12.
- 122. chapter 6 Design for Usability Symposia 120 Design for Usability Methods Tools
- 123. 2009: Design for Usability 2010: Product Impact 2011: Methods Tools The aim of the Design for Usability project was to help product development practitioners create usable products, so we have spent a great deal of time and effort on disseminating the results of our research programmes. One of the platforms which we chose were the yearly Design for Usability symposia on World Usability Day. The first World Usability Day was held in 2005. It was introduced by the Usability Professionals’ Association to ensure that services and products important to human life are easier to access and simpler to use. On this day around the world, professionals in the field engage in discussions on the tools and issues central to excellent usability research, development and practice (www. worldusabilityday.org). A second aim of the day is to raise the general public’s awareness of usability and of its importance. Throughout the DfU project, each year on World Usability Day, we hosted a symposium with presentations by practitioners and researchers, and with workshops in which the latest results of the Design for Usability project could be experienced hands-on. Of course the day also became an enjoyable venue for usability and user experience practitioners and researchers to meet and discuss. The 2011 DfU Symposium was the last one that fell within the duration of the Design for Usability project. However, because the response from the field was so positive, we decided to organize a 2012 symposium as well. Again, it featured presentations by practitioners and workshops, and in addition, the book that you are holding now was presented. At the time of writing, because of the positive experiences and feedback from the previous symposia, we are exploring the options for continuing the Design for Usability symposium as an annual event where practitioners and researchers can continue to exchange the latest knowledge and insights on usability, user experience and user-centred design. On World Usability Day, of course. 121
- 124. 122 Design for Usability Methods Tools Symposium 2009 Design for Usability
- 125. 123
- 126. 124 Design for Usability Methods Tools DfU Symposium 2009 Design for Usability Delft University of Technology The day concluded with a reception held at the Design United exhibition. Here participants were able to meet fellow user-centred product development professionals and academics, and reflect on a day packed with exciting new insights in the field of usability. In 2009, the DfU project team organized the first DfU project symposium. The chance to find out more about state-of-the- art usability clearly struck a chord with product development practitioners and researchers, as the 300 seats available were sold out within three weeks. DfU project leader Daan van Eijk welcomed the attendees and introduced the DfU project, after which the symposium kicked off, content-wise, with four presentations by usability practitioners and researchers. After lunch things got more interactive, when the participants spread out all over the building to join the workshop of their choice. The workshops, given by the university researchers and representatives from the companies involved in the DfU project, had been specially crafted so that both novices and professionals could join a workshop of their liking. After the presentations, the deans of the Industrial Design Engineering faculties of the three Dutch universities of technology opened the Design United exhibition. This yearly event showcases excellent designs by students of three university-based Dutch Industrial Design Engineering schools. In 2009 the theme of the exhibition was ‘Design for Usability’.
- 127. 125 Symposium 2009 7 Toolbox for Usability – Learn about the MUST tool: managing usability information supporting decision- making Christelle Harkema Cha Joong Kim 8 Usability in practice – barriers and enablers Jasper van Kuijk Proceedings Abstracts of the lectures and workshops were published in the DfU 2009 symposium proceedings which were sent to everyone who registered for the symposium. More information Symposium proceedings An extensive and highly illustrated report of this exciting day: http://bit.ly/dfu-symp09-book Symposium videos View the presentations online http://bit.ly/dfu-symp09-vid Presentations Designing for a moving target – from functionality to usability to experience By Gerrit C. van der Veer – Open University Netherlands and University of Sardinië, President ACM SIGCHI Usability in a productive print environment By Abbie Vanhoutte Robert Eijlander – Océ- Technologies B.V. The challenges in interaction design for consumer and professional electronics By Cees van Dok – Frog Design Europe No silver bullet – Why making usable consumer electronics requires organizational change By Jasper van Kuijk MSc – TU Delft Workshops 1 The User Centred Experience – Experiencing a typical ‘User Centred Design’ cycle in an interactive workshop Abbie Vanhoutte Robert Eijlander – Océ-Technologies B.V. 2 Managing Design for Usability in practice – Professional debate on effectively addressing usability risks and opportunities. Willem Mees van der Bijl - Indes 3 Advanced user research and evaluation – How to get the best and most out of user research and evaluation Roel Kahmann – P5 Consultants 4 Guiding and changing user behaviour – Improving design by learning to assess how products change users Steven Dorrestijn 5 The usability runway – a practical introduction in approaching usability related design assignments Mascha van der Voort, Irene Anggreeni Frederik Hoolhorst 6 Small usability techniques – Practical application and added value of ‘small usability techniques’ in design Mieke van der Bijl-Brouwer Stella Boess
- 128. 126 Design for Usability Methods Tools Symposium 2010 Product Impact
- 129. 127
- 130. 128 Design for Usability Methods Tools DfU Symposium 2010 Product Impact University of Twente, Enschede concluded with a panel discussion about the possibilities, but also about the practical and ethical problems concerning the integration of product impact in design. In addition to the presenters, the panel included four eminent professors: Peter Paul Verbeek, Paul Hekkert, Wim Poelman and Timo de Rijk. Presentations Mediation theory and design By Peter-Paul Verbeek – University of Twente Design for social behaviour By Nynke Tromp – Delft University of Technology On the Design with intent toolkit By Dan Lockton – Brunel University London, UK Product impact in the design of the ‘OV-chipkaart By Steven Dorrestijn – University of Twente Workshops 1 Redesigning the ‘OV-chipkaart using Product Impact theory Peter-Paul Verbeek Steven Dorrestijn The central theme of the symposium held on World Usability Day 2010 was ’Product Impact’. This symposium aimed at exchanging scientific knowledge about product impact with the 130 participants. In the morning, four speakers preented on the theory, ethics, and design of behaviour-influencing technologies. In the afternoon, workshop participants learned about tools and participated in interactive sessions about guiding and changing user behaviour through design. The afternoon
- 131. 129Symposium 2010 Panel discussion http://bit.ly/dfy-symp10-pan 2 Experiencing the Design with Intent Toolkit Dan Lockton 3 Design for Social Behavior Paul Hekkert Nynke Tromp More information The video recordings of this symposium can be viewed online: Peter-Paul Verbeek: Mediation theory and design http://bit.ly/dfu-symp10-ver Nynke Tromp: Design for Social Behavior http://bit.ly/dfu-symp10-tro Dan Lockton: Design with Intent Toolkit http://bit.ly/dfu-symp10-loc Steven Dorrestijn: Product impact in the design of the ‘OV-chipkaart http://bit.ly/dfu-symp10-dor
- 132. 130 Design for Usability Methods Tools Symposium 2011 Methods and Tools
- 133. 131
- 134. 132 Design for Usability Methods Tools DfU Symposium 2011 Methods and Tools Media Plaza, Utrecht themselves with new DfU methods and tools. The day concluded with a preview of the Design for Usability book and website through which all the tools and methods created during the DfU project will be made accessible to practitioners and researchers. Presentations Usability at Philips By Bert Ipema – Philips Consumer Lifestyle Drachten Usability at Indes By Willem Mees van der Bijl – Indes Transfer of the results of the DfU project By Jasper van Kuijk – Design for Usability project Workshops 1 The UCD Kick-off tool – Creating a plan of approach for your user-centred product development process Frederik Hoolhorst Mascha van der Voort During this third DfU symposium, the usability tools and methods developed during the DfU research projects were presented practically and effectively. The event formed the concluding event of the DfU research project, and, fittingly, was held at the inspiring Media Plaza in Utrecht. In the morning, 200 participants listened to presentations by multinational Philips and design consultancy Indes on how user centred product development has changed within both companies over time. The afternoon program consisted of 6 interactive workshops where participants could familiarize World Usability Day 2011 Thursday November 10 10.00 Doors open 10.30 Design for Usability lectures Three inspiring lectures about design for usability at multinational Philips, design consultancy Indes and the 3TU research project 13.00 Lunch 14.00 Methods tools workshops Participate in sessions to learn to work with new methods tools to design for usability 16.30 Preview Design for Usability Workbook 17.00 Drinks Update Usability Knowledge Symposium Usability Methods tools Utrecht. November 10, 2011
- 135. 133Symposium 2011 2 Improving usability decision-making – How to prevent unawareness in your design practice Christelle Harkema Ilse Luyk-de Visser 3 Interaction between Product and User profiles – How to increase users’ satisfaction and avoid mistakes Cha Joong Kim Henri Christiaans 4 Product Impact Tool – How can user behaviour be guided and changed through design? Steven Dorresteijn Peter-Paul Verbeek 5 Introducing the Design for Usability Method Selection Tool – How to find the best method for your project Tristan Weevers Jaap Daalhuizen 6 The Envisioning Use workshop – A team technique for early development to create a common vision on product use Stella Boess Mieke van der Bijl-Brouwer
- 136. chapter 7 Related projects 134 Design for Usability Methods Tools
- 137. 7.1 Managing Soft Reliability 7.2 Embedded and Embodied Usability 7.3 REPAR The DfU project was setup to stimulate DfU awareness and provide product design and development practitioners with new and practical information about usability and user- centred design. This project is, however, not the only source of information on the subject, so in Chapter one we included an overview of the relevant literature. This chapter adds to this by highlighting three related Dutch research projects, two of which were funded by the same IOP research initiative as the Design for Usability project, namely Integrated ProductCreation and Realisation (IPCR). 7.1 The IOP-IPCR Managing Soft Reliability project resulted in three software tools that allow product developers to ensure their products match user requirements and expectations (i.e., soft reliability). These tools provide product developers with the opportunity to collect and analyse user experience data, do data mining on product development processes, and retrospectively elicit longitudinal user experience data. 7.2 Embedded and Embodied Usability was a collaboration between practitioners and researchers which identified success factors for user-centred design in the context of complex, distributed and multidisciplinary product development. 7.3 The IOP-IPCR REPAR project was initiated to resolve the paradox that designers would like to know at an early stage whether their ideas and concepts will provide value to people, but that participants find it hard to assess this in early evaluations, when perhaps only sketches and words are available. The project resulted in tools to create preliminary concept representations and methods to explore these together with users. 135 chapter xx Xx
- 138. 136 Design for Usability Methods Tools user preferences, routines, and contexts of use. However, creating that intelligence requires designers to deal with a large degree of uncertainty, because how these products are actually used and adopted in the field is often unknown. Although conventional reliability techniques typically succeed in safeguarding the reliability of hardware and software (i.e., the products work as specified and are stable from a technical point of view), users increasingly complain about ‘non-functioning’ products. Consequently, while it is possible to develop these products to match technical specifications, i.e., hard reliability; it is not clear how to match the diverse user requirements and expectations, i.e., soft reliability. The major proportion of current product rejections tend to be related to soft reliability issues. Research method The project team conducted multidisciplinary research following the ‘industry as a laboratory’ philosophy to ensure that the results are not only innovative from an academic perspective, but also address significant real-life problems and are practically applicable. Through case studies, and identifying the information needs of stakeholders, the Author bio Aylin Koca was born in Ankara, in 1980. She studied at Bilkent University and graduated with a BSc and MSc in Computer Engineering. She was awarded her PhD in Industrial Design at Eindhoven University of Technology. She is the cofounder of the award-winning startup UXsuite. In addition, she is a postdoctoral researcher at TU/e. 7.1 Managing Soft Reliability Data analysis tools for improving user experiences Aylin Koca Introduction A large range of electronics products, from smartphones to medical equipment, from multifunction office printers to cars, increasingly have context-aware and adaptive features. They display a form of intelligence to better comply with diverse
- 139. 1377.1 Managing Soft Reliability Fluxicon’s process mining solutions enable fast and easy process visualization and analysis. Its innovative process mining technology uses unstructured, event-level data to automatically build process definitions and models, and to explore process variations. Due to the quantitative nature of the data, the derived process models enable rich and interactive analysis. iScale, helps to highlight soft reliability issues where users have trouble truly incorporating the product into daily life. It has proven a viable and lightweight retrospective elicitation method in comparison to the expensive longitudinal methods. Project specs Team members Mathias Funk, Evangelos Karapanos, Aylin Koca, Anne Rozinat Duration 2005-2010 More information Project homepage Managing Soft Reliability Research Project www.softreliability.org UXsuite Software suite for UX data collection, analytics, and experience crafting www.uxsuite.com Fluxicon Software tools and services for business process mining and analysis www.fluxicon.com requirements for fast and flexible field-feedback mechanisms were identified. Accordingly, solutions were designed, prototyped and successfully evaluated at DfU project partner companies, Philips Consumer Lifestyle and Océ. Results The knowledge and tools generated in this project have been transferred to and applied by industry with great success, resulting in two software startups, namely UXsuite and Fluxicon, as well as in the user experience survey tool iScale. UXsuite is an integrated software suite for real-time collection and visual analysis of both qualitative and quantitative user experience data from products as well as websites. Usage patterns of interest can be defined and revised at any point in time seamlessly, while these patterns can also be used to trigger certain actions (e.g., automatic referral of the user to another component of the interface, custom tailoring). Fluxicon is a set of software tools and services for business process mining and analysis, compliance, and improvement. iScale is a survey tool for the retrospective elicitation of longitudinal user experience data. It is designed to increase the effectiveness of recalling product experiences, eliciting changes in product perception and evaluation over time. It aims to minimize retrospection bias and employs sketching to impose a process during the reconstruction of a consumer’s experiences. Innovation benefits With UXsuite technology, field feedback time of electronics products is dramatically reduced from 1.5 years to real-time. Moreover, the logic that steers both data collection and analysis can be dynamically aligned with evolving business objectives, providing flexibility and ensuring relevance of data.
- 140. 138 Design for Usability Methods Tools as systems are composed of a range of products, software applications and services. This study aims to distil success factors for enhancing user-centred design from the complex, distributed and multidisciplinary product development at Océ. Research Method In this project, development practitioners from Océ are joined by external researchers in an attempt to better understand their own practice. It is an inquiry into product development ‘in-the-wild’, blending theories and observations. Data included interviews with developers and stakeholders; a range of workshops with designers and developers; over 30 hours of filmed meetings; and participatory observations. Experiments were also conducted to validate the insights gained. Findings Multidisciplinary product development includes many experts with distinct knowledge and skills. It is often difficult for teams to integrate all the experts’ activities seamlessly into a coherent whole, whereby the contribution of all team members impacts the usability and user experience of 7.2 Embedded and embodied usability Success factors for user-centred design in complex product development Guido Stompff Author bio Guido Stompff bridges the world of design practice and design research. He is senior designer at Océ Technologies, part of the Canon Group. His study of design and innovation in-the-wild, resulted in him being awarded a PhD in 2012 for his thesis: Facilitating Team cognition. How designers mirror what teams do. Introduction In our digital and networked society, high tech systems offer increasing functionality while human abilities and skills to cope with complex tasks essentially remain the same. To empower users, the challenge is to design intelligent systems that offer a simple and enjoyable interaction. Unfortunately, developing systems is also becoming increasingly difficult,
- 141. 139 7.2 Embedded and embodied usability products, either directly or indirectly. Thus, usability is not something that can be done separately from the activities of these experts; rather it is a result of collective efforts. Success factors are: Think ‘prototypes’ No one can truly oversee beforehand what the impact is of the efforts of many developers working together on the eventual product. Only by means of building and testing integrated prototypes, can teams learn ‘on-the-go’ what the impact is of their choices on usability and what is possible. Talk ‘stories’ and ‘visuals’ Language is generic and thus an impoverished means for non-existing products. Requirements may be comprehensive, but provide no clue as to what the ‘big picture’ is. Therefore vivid and compelling representations of the intended system are required, for example stories and visualizations. Experience ‘the real thing’ The experience of interacting with tangible, real world objects enables teams to reconcile contesting aims. Seeing how something works, how it sounds or how it feels: these experiences are relatively the same for all. Actively deploying prototypes, models or demonstrators in meetings are highly beneficial for usability. Seeing ‘the user’ Usability testing or customer trials that involve team members are highly beneficial for the resulting usability of products. Those involved in these tests see how users struggle with the artefacts of their work. They observe how they can adapt their work to enhance usability, without even the need for discussion. Innovation benefits This study’s findings are extremely practical and suggest a paradigm shift for developing usable products. The study conceives usability as an integral part of development, engineering and design, rather than something separate. Usability is not the sole responsibility of designers or usability experts; it needs to be deeply embedded in the practice of product development. Project specs Team members Guido Stompff (Océ/TUD), Fred de Jong (Océ) and Eddy van Vliembergen (Océ), Lilian Henze (P5 consultants), Pieter-Jan Stappers (TUD), Frido Smulders (TUD), Jan Buijs (TUD) and Jo Geraedts (Océ/TUD). Duration August 2009 – December 2010 Figure 1 : The product that served as the context for the research: the Océ VarioPrint DP line. It is launched end 2011. Figure 2: A usability test in the wild: team members observe a usability test themselves.
- 142. 140 Design for Usability Methods Tools Introduction Designers would like to know at an early stage in the design process whether their ideas and concepts will make sense to people, that is, whether the concepts, if turned into products or services, will provide value to people in their everyday lives. However, for end users it is often difficult to give reliable evaluations about whether these early concepts will add value, because the preliminary nature of the concepts makes it difficult for them to anticipate the eventual user experience. For this reason, users are often not involved in the concept development phase of a design project, but only in the early (user research) and late (evaluation) phases. Result The REPAR project will provide four end products. Two tools, IdAnimate and Sketchify are being developed, which will provide designers with a simple means to create and explore low-fidelity prototypes, appropriate for the early stages of the design process. The project will also add to the understanding of how Virtual and Augmented Reality technologies can be used to create more high-fidelity experiential prototypes, appropriate for later stages of the concept development process. The project will also result in a new methodology, the Co- Constructing Stories method, which enables designers to discuss early concept representations with end users in interview-like sessions. A session consists of two phases, 7.3 REPAR Resolving the Paradox in User-centred Design through Flexible Prototyping Jacques Terken Author bio Jacques Terken is a cognitive psychologist by training. His expertise is in the area of human-computer interaction. His research interests include user experience methodology and automotive human factors.
- 143. 1417.3 REPAR a sensitization phase in which past experiences are elicited from end users, and an elaboration phase in which designers elicit anticipated future experiences from end users about a proposed concept, resulting in a story about how the concept will provide value to end users. The concept representations are created with IdAnimate, Sketchify or Virtual Reality tools, depending on the phase of the design process. Innovation benefits The REPAR project aims to provide low-threshold tools and methods for designers, facilitating the inclusion of end users in the phases of the design process where crucial design decisions are made. Project specs Team members Derya Ozcelik (TUE), Javier Quevedo-Fernandez (TUE), Jos Thalen (UT), Jean-Bernard Martens (TUE), Mascha van der Voort (UT), Jacques Terken (TUE) Duration 2009 – 2013 More information REPAR project homepage www.repar-project.com Figure 1: A design team in action using IdAnimate Figure 3: A Co-constructing Stories session Figure 2: The use of Augmented Reality for concept exploration
- 144. 142 Design for Usability Methods Tools Design for Usability Publications
- 145. 143 Publications Theses Van der Bijl-Brouwer, Mieke (2012) Exploring Usability, design for dynamic and diverse use situations. PhD thesis. University of Twente, Enschede, The Netherlands. Dorrestijn S. (2012). The design of our own lives: Technical mediation and subjectivation after Foucault. PhD Thesis. Twente University, Enschede, The Netherlands. Harkema, C.L.E. (2012). Revealing unawareness in usability related decision-making. PhD Thesis. Technische Universiteit Eindhoven, The Netherlands. Hoolhorst, F. W. B. (2012) Structuring user-centred product development processes. PhD thesis. University of Twente, Enschede, The Netherlands. Kim, C.J., (2012) Anticipating soft problems with consumer electronic products: How do soft problems interact with user characteristics and product properties? PhD Thesis. Delft Univeristy of Technology, Delft, The Netherlands. Kuijk, J.I. van (2010) Managing Product Usability: How companies deal with usability in the development of electronic consumer products. PhD Thesis. Delft University of Technology, Delft, The Netherlands. Weevers, T.J.T. (2011) ‘Method selection tool for user- centred product development’ MSc thesis, Delft University of Technology, Delft, The Netherlands. Designs TRACE: interactieve software for analyzing barriers and enablers in product development. Visualizes an XML database of barriers and enablers generated through the Atlas.ti qualitative data analysis package. Beta version. Designer: J.I. van Kuijk UCDtoolbox; online resource for selecting user- centered design methods. Alpha version. Designer: T.J.T. Weevers Peer reviewed publications Bijl-Brouwer, M. van der, and Voort, M.C. van der, “Strategies to design for dynamic usability” in proceedings of IASDR2009 Rigor and Relevance, 18-22 October 2009, Seoul, South-Korea. Bijl-Brouwer, M. van der, and Boess S.U., “From remembering to envisioning product use: an informal design technique”, 7th International Conference on Design Emotion, October 4-7 2010 Chicago, USA. Bijl-Brouwer, M.van der, “Exploring design for dynamic use” accepted for the 13th International Conference on Engineering and Product Design Education, City University, London, UK, 8-9 September 2011 Bijl-Brouwer, Mieke van der, Stella Boess and Christelle Harkema, “What do we know about product use, a
- 146. 144 Design for Usability Methods Tools Kim, C.J. and Christiaans, H.H.C.M. (2009). User profiles based on soft usability problems in consumer electronic products. In K Lee, J Kim LL Chen (Eds.), Proceedings of the IASDR2009 “Design Rigor Relevance” (pp. 1-11). Seoul: Korean Society of Design Science. Kim, C.J. and Christiaans, H.H.C.M. (2011) Product Profile to Reduce Consumer Dissatisfaction in terms of Soft Usability Problem and Demographical Factors: an Exploratory Study. ICED ”11. Copenhagen, Denmark. Kim, C.J. and Christiaans, H.H.C.M. (2011) Who Are The People That Experience Soft Usability Problems? HCI International 2011. Orlando, USA. Kim, C.J. and Christiaans, H.H.C.M. (2011). “Soft” Usability Problems with Consumer Electronics: The Interaction between User Characteristics and Usability. Journal of Design Research. Kim, C.J. and Christiaans, H.H.C.M. (2011). Usability problems: the influence of user diversity. IASDR 2011. Delft, The Netherlands. Kuijk, J.I. v., H.C.C.M. Christiaans, H.Kanis, D.J. van Eijk. (2007). Usability in Product Development Practice: After Sales Information As Feedback. Conference of the International Association of Societies of Design Research: Emerging Trends in Design Research. November 12-17 2007, Hong Kong, China. Wever, R., Kuijk J.I. van, Boks, C. (2008) User-Centered Design for Sustainable Behaviour. Journal of sustainable Engineering, Volume 1, Issue 1. Taylor Francis, London Kuijk, J.I. van, L. van Driel, D.J. van Eijk (accepted with revisions) Usability in practice; a cross-market comparison. Journal of Applied Ergonomics. Elsevier Science. Kuijk, J.I. van, H. Kanis, H.H.C.M Christiaans, D.J. van Eijk (accepted with revisions) Barriers and enablers for usability in practice – A multiple case study of product development groups of electronic consumer products Mulders, E., Sonnemans, P.J.M., Geudens, W.H.J.M., technique to share use-related knowledge in design teams.” IASDR2011, the 4th World Conference on Design Research, 31 october – 4 November, Delft, 2011 Eijk, D. van, van Kuijk, J., Hoolhorst, F., Kim, C., Harkema, C., Dorrestijn, S. (2012), Design for Usability; practice-oriented research for user-centered product design. In : Work: A Journal of Prevention, Assessment and Rehabilitation (Supplement 2012/1: IEA 2012: 18th World congress on Ergonomics – Designing a sustainable future), 41 (0), pp. 1008-1015. Harkema, C.L.E., Luyk-de Visser, I.M., Sonnemans, P.J.M. Brombacher, A.C. “Can existing usability techniques prevent tomorrow’s usability problems? Submitted for ICED, Copenhagen, 2011 Hoolhorst F.W.B. and Voort, M.C. van der, (2009) A concept for a usability focused design method, IASDR 2009, Seoul Hoolhorst, F.W.B. and Voort, M.C. van der (2009). Criteria for new-generation user-centered design methods. Design Principles and Practices: An International Journal, 2009. 3(1): p. 145-154 Kim, C.J., Christiaans, H.H.C.M., Eijk, D. van. (2007). Soft Problems in Using Consumer Electronic Products. International Conference of IASDR, Hong Kong Kim C.J. Christiaans, H.C.C.M., (2008) The Correlation Between Soft Problems and User Characteristics. Proceedings of SEAES conference Kim, C.J. Christiaans, H.H.C.M. (2009). Usability and ‘Soft Problems’: a conceptual framework tested in practice”, at the Conference of the International Ergonomics Association (IEA ‘09) in Beijing (August 9-14 2009). Kim, C.J. and Christiaans, H.H.C.M. (2009). ‘Soft’ problems with consumer electronics and the influence of user characteristics. Proceedings of the 17th International conference on engineering design, ICED’09 (pp. 241-251). S.l.: The Design Society.
- 147. 145Publications through design?” In: Daan van Eijk (ed.) Symposium Design for Usability 2009. Delft: IOP IPCR Design for Usability Research Project, pp. 76-79. Dorrestijn, S. (2010) “Design your own life: Over ethiek en gebruisvriendelijk ontwerpen”. In: M. Huijer M. Smits (red.), Moralicide: Nieuwe morele vocabulaires voor technologie. Kampen: Klement, pp. 90-104. Dorrestijn, S. (2010) “Boekbespreking: Niet vergeten uit te checken!” (Recensie van: Christian van ”t Hof, Rinie van Est, Floortje Daemen (red.) Check in / check uit. De digitalisering van de openbare ruimte. Rotterdam, Nai Uitgevers, 2010). In: Filosofie Praktijk, 31/3, pp. 88-91. Dorrestijn, S. (2011) “Gedragsbeïnvloedende techniek en usability”. In: Tijdschrift voor Ergonomie 36-1, pp. 5-12. Hoolhorst, F.W.B., “Ergernomie – Mag het een knopje meer zijn?”. In: Tijdschrift voor Ergonomie 35-2 (2010), pp. 31. Hoolhorst, F.W.B., Symposium product impact. In: Tijdschrift voor Ergonomie 35-6 (2010), pp. 26-27 Hoolhorst, F.W.B.: ”Scenario based product design”. Lecture at the University of Twente, November 9, 2010 Hoolhorst, F.W.B. “Experiencing the new usability-centred design method”. Workshop at the 7th IOP-IPCR-DfU BC Meeting, Utrecht, March 2011 Hoolhorst, F.W.B. Dorrestijn S. (2011), Dossier: Design for usability. Tijdschrift voor Ergonomie 36(1): 4-17 Kuijk, J.I. van, Christiaans, H.C.C.M., Kanis, H. and Eijk, D.J. van (2007). Usability in Product Development: A Conceptual Framework. The British Ergonomics Society Annual Conference April 17-19 2007, Nottingham, UK. Kuijk, J.I. van (2007). Waarom Werkt het niet Gewoon? Gebruiksgemak in het Productontwikkelingsproces. Product Magazine, Media Business Press, Rotterdam. Tijdschrift voor Ergonomie. Nederlandse vereniging voor ergonomie. ISBN 0921-4348. Kuijk, J.I. van (2008) Maak het jezelf makkelijk: in 8 stappen naar een gebruiksvriendelijk product. Tijdschrift Brombacher, A.C. (2010). Increasing customer complaints – insights how to reduce them. Submitted for publication in Reliability Engineering and System Safety. Sonnemans, P.J.M., Mulders, E., Christiaans, H.H.C.M., Newby, M.J., Brombacher, A.C. (2010) Customer complaints about consumer products in different cultures. Submitted for publication in Quality and Reliability Engineering International. Other scientific publications Dorrestijn, S., “Design and Ethics of Product Impact on User Behavior and Use Practices”. In: M. Schneider et al. (ed.), Workshops Proceedings of the 5th International Conference on Intelligent Environments: Volume 4. Ambient Intelligence and Smart Environments. Amsterdam: IOS, 2009, pp. 253-260. Verbeek, P.P., “Let’s Make Things Better: Persuasive Technology and Material Morality”. Paper in session “Styles and Skills in the Culture of Design”, organized by the Netherlands Graduate School for science, Technology, and Modern Culture. 2008 Conference of the Society for the Social Studies of science. Rotterdam, August 2008 (invited paper, unpublished) National or local (scientific) journals Dorrestijn, S. en Mul, J. de, “China uit de tijd. Gesprek met Jos de Mul over het Rijk van het Midden”. In: Filosofie Praktijk 28-5 (2008), pp. 23-28. Dorrestijn, S. (guest editor), Wijsgerig Perspectief (special issue: De actuele Foucault) 49-2 (2009). Dorrestijn, S., “Ten geleide”. In: Wijsgerig Perspectief 49-2 (2009), pp. 4-5. Dorrestijn, S., “Filosofie met het eigen bestaan als inzet. Inhoud en actualiteit van Foucaults late werk”. In: Wijsgerig Perspectief 49-2 (2009), pp. 30-37. Dorrestijn S. Tromp, N. (2010), “Design for behaviour. How can user behaviour be guided and changed
- 148. 146 Design for Usability Methods Tools voor Ergonomie Jaargang 33, 2008 : nummer 1, februari. Kuijk, J.I. van, Preijde, E.E., Toet, E.N., Kanis, H. (2009) Expected versus experienced usability: what you see is not always what you get. IEA2009, 17th World Congress on Ergonomics, August 9-14, 2009, Beijing, China. (Presentation and publication in conference proceedings). Oral presentations Dorrestijn, S. “Technical Mediation and Subjectivity. The Missing Link between the Morality of Artifacts and Moral Philosophy”. Presentation at the biannual conference of the Society for Philosophy and Technology, Charleston, South Carolina, US, July 2007. Dorrestijn, S., “Product impact and Usability. Project overview.” Presentation at the Behavioral Research PhD meeting, University of Twente, November 2007. Dorrestijn, S., “Product Impact for Usability”. Presentation at the Multidisciplinary Workshop Twente- Texas-Colorado, Enschede, May 2008. Dorrestijn, S., “Technical mediation and subjectivity. Human-technology interactions in art and technology design”. Presentation at the annual conference of the Society for the Social Studies of Science (and EASST). Rotterdam, August 2008. Dorrestijn, S., “Product Impact. Technical Mediation in Philosophy and Design”. Presentation at the WTMC Summer School, Ravenstein, August 2008. Dorrestijn, S., “Utopie en design: Sociale verandering en techniekontwerp”. Paper voor de Nederlands-Vlaamse filosofiedag, Leuven, November 27, 2008. Dorrestijn, S., “Future Philosophy. Philosophy of the future”. Presentation at the Philosopher”s Rally. Enschede, May 2009. Dorrestijn, S., “De ethiek van het wennen aan techniek. De ethiek van de techniek in de zorg volgens Vorstenbosch”. Presentation and debate participation concerning: Jan Vorstenbosch (2009), Hoe maakt u het? Technologie in een veranderende gezondheidszorg: Over dossiers, robots en tests in de zorg. Den Haag: ZonMW. Den Haag, June 2009. Dorrestijn, S., “Governing and Fashioning the Subject: Extending Foucault”s Philosophy of Technology”. Presentation at the Foucault 25 years later conference. Rotterdam, June 2009. Dorrestijn, S., “Utopia and Design. Social change by means of technology”. Presentation at the biannual conference of the Society for Philosophy and Technology, Enschede, July 2009. Dorrestijn, S., “Theories and figures of technical mediation”. Presentation at the annual conference of the Society for the Social Studies of Science. Washington DC, USA, October 2009. Dorrestijn, S. “Introduction to Research Project”, participation in Technoscience seminar, Stony Brook University, US, June 12-14, 2010. Dorrestijn, S. “Utopian Design”. Paper presentation at Politics of Design Workshop, Manchester, UK, June 24, 2010 Dorrestijn, S., “Product Impact and the design of the OV- chipkaart”, Presentation at the DfU Symposium Product Impact, Enschede, November 11, 2010. Dorrestijn, S. “De omvorming van onszelf in filosofie, kunst en techniek. Bij het afscheid van Petran Kockelkoren. Presentation at Kockelkoren Symposium, June 2011. Hoolhorst, F.W.B., “Criteria regarding new-generation user-centered design methods”. Presentation at the “Third international conference on design principles and practices” (Berlin; 15-17 February 2009). Hoolhorst F.W.B. “A concept for a usability focused design method”. Presentation at the IASDR 2009, Seoul, October 2009.
- 149. 147 Publications Hoolhorst, F.W.B. (2010). Design methodology. PhD@Sea. June 10-11 2010, Egmond, The Netherlands (Presentation and publication in conference proceedings) Kim, C.J. (2008). Presentation given at the Design research workshop at the departments of industrial esign at Yonsei University and Korean University of Technology in South Korea in August 2008. Weevers, T.J.T. (2011). Method Selection for User Centred Product Development. Master thesis defence , August 31, 2011. Delft, Netherlands. (Presentation and publication via designforusability.org). Invited lectures Dorrestijn, S. (with Verbeek, P.P.C.C.) “Product Impact”. Lecture at the Design Academy, Eindhoven, October 1 2008. Dorrestijn, S., “Technical mediation in philosophy and design. Transferring some STS concepts to the practice of design”. Invited lecture in a course on Science and Technology Studies at the Vrije Universiteit Amsterdam, September 10, 2009. Dorrestijn, S., “De omvorming vatn het zelf in filosofie, kunst, en techniek”. Invited lecture at conference “Michel Foucault: Veroog en politiek”. Antwerpen, Lessius Hogeschool, November 18, 2009. Dorrestijn, S., “Technical mediation in philosophy and design: Dealing with how technologies change us”. Lecture at the Design Academy, Eindhoven, March 17, 2010. Dorrestijn, S. “Filosoferen in de trein: OV-chipkaart en privacy”. Invited lecture at Studium Generale, Technical University Eindhoven, September 15, 2010. Dorrestijn, S. “Technical mediation in philosophy and design. Transferring some STS concepts to the practice of design”. Invited lecture in a course on Science and Technology Studies at the Vrije Universiteit Amsterdam, September 16, 2010. Hoolhorst, F.W.B.: ”Scenario based product design”. Lecture at the University of Twente, November 9, 2010 Kim, C.J.: “Soft problems in product use” Lecture at Yonsei University and Korean University of Technology in South Korea, August, 2008 Kim, C.J. “New consumer complaints in electronic industry and user characteristics”. Lecture at Korean University of Technology in South Korea, August, 2009 Kuijk, J.I van: “Innovatie en productgebruik”. Invited lecture at KPN innovation workshop for 70 managers involved with innovation within KPN. September 13, 2007, The Hague, The Netherlands. Kuijk J.I. van: “Everything but testing; an organizational- and process-wise approach to usability”. Invited presentation for a meeting of the Dutch Design Management network. November 4, 2008. Kuijk J.I. van: “Usability and consumer products”. Presentation at the Dutch Consumers” Association (Consumentenbond). October 30, 2008, The Hague, The Netherlands. Kuijk J.I. van: “User-centered design; A reality Check” Keynote presentation at CHI*Sparks 2011 conference. June 23, Arnhem, the Netherlands. Kuijk J.I. van: “Human-centered design; De Harde Realiteit” Invited presentation at symposium ”10 Years Human Technology”, Hanze Hogeschool. May 19 2011, Groningen, The Netherlands. Kuijk J.I. van: “Managing Product Usability” Invited presentation, Philips Consumer Lifestyle. March 23, 2011, Drachten, The Netherlands. Interviews Keuning, W. (2010) “Interview promovendus Jasper van Kuijk. Gebruiksgemak elektronica ver te zoeken” In: De Volkskrant, 15 december 2010 (interview met J.I. van Kuijk) Penris, I. (2010) “Doctor in de rot-apparaten kiest toch
- 150. 148 Design for Usability Methods Tools voor het cabaret”. In: Algemeen Dagblad, 14 december 2010 (Interview met J.I. van Kuijk) Monteri, A. (2010) “Gebruiksgemak vaak stiefkind”, In: De Telegraaf, 24 december 2010 (Interview met J.I. van Kuijk) Aan de slag (2010) BNR radio, 13 december 2010, 11:15 (am) (Interview met J.I. van Kuijk) Radio 1 Journaal (2010) Radio 1, 13 december 2010, 8:15 (am) (interview met J.I. van Kuijk) Tijd voor Twee (2010) Radio 2, 13 december 2010, 12:45 (interview met J.I. van Kuijk) Start (2010) FunX radio, 13 december 2010, 7:15 (interview met J.I. van Kuijk) NOS Journaal (2010) Nederland 1, 11 december 18 uur, 20 uur (interview met J.I. van Kuijk) Tros Nieuwsshow (2010) Radio 1, 18 december, 9:45 (am) (interview met J.I. van Kuijk) Bueters, P. (2010) Gadgets Snappen U Niet / Hoe Maak je Gadgets Gebruiksvriendelijk? In: website Management Team magazine / Techbusiness, 15 december 2010 (interview met J.I. van Kuijk) Hoe?Zo! Radio (2010) Radio 5, 15 december, 20:30 (interview met J.I. van Kuijk) Tros Radar (2010) Gebruiksgemak is niet vanzelfsprekend. Tros Radar Website, 24 december 2010 (interview met J.I. van Kuijk) Berg, M. van den (2011) Gemakzucht; Gebruiker Centraal Tijdens Ontwerpproces (Interview met J.I. van Kuijk. De Ingenieur Nr. 1, Jaargang 123, 14 januari 2011. Striens, R (2011) Update - Jasper van Kuijk: Man met een Missie. In: Items #1 (Interview met J.I. van Kuijk) BNR Nieuwsradio (2011) Interview met J.I. van Kuijk. 11 augustus 2011, 8:15 am. Felix Wadewitz (2011) Coverstory: Einfach einfach (contains interview with J.I. van Kuijk). Impulse; Das Magazin für Unternehmer (contains interview with J.I. van Kuijk), August 2011. G+J Wirstschaftsmedien AGCO. KG Germany. Giesen, P. (2008). De chip is meer mens geworden. In: De Volkskrant, Kennis, 30 Aug 2008 interview met P.P.C.C. Verbeek) Pous, I. de (2008), “Technologie verandert onze waarden”. In: Trouw, 17 maart 2008 (interview met P.P.C.C. Verbeek) Verbeek, P.P.C.C. (2008). Bijdrage aan radiodiscussie over techniek en ethiek, Desmet Live, 28 maart 2008 Krijnsen, M. (2008). Waarom de hazenlip zeldzaam wordt. In: Twentsche Courant Tubantia, 31 Aug 2008. (interview met P.P.C.C. Verbeek) Workshops Boess, S. and Bijl-Brouwer, M. van der. “Small usability techniques”. Workshop at the World Usability Day 2009, Delft, November 2009. Bijl-Brouwer, M. van der., Boess, S, Harkema, C.L.E. “Knowledge Gathering”. Workshop at Van Berlo, Eindhoven, November 2010 Bijl-Brouwer, M., introduction Envisioning Use workshop for new DfU companies at Utrecht, 24th March 2011 Bijl-Brouwer, M., Envisioning Use workshop at Indes, Enschede, 31 March 2011 Bijl-Brouwer, M. and Boess, S. “Envisioning Use “ workshop at Oce, April 2011 Bijl-Brouwer, M. and Boess, S. “Envisioning Use “ workshop at Philips Consumer Lifestyle, May 2011 Bijl-Brouwer, M. and Boess, S., Harkema, C.L.E. “Envisioning Use “ workshop at Philips Consumer Lifestyle, August 2011 Dorrestijn, S. and Tromp., N.: “Guiding and changing user behavior”. Workshop at the World Usability Day 2009, Delft, November 2009. Dorrestijn, S. “Guiding and changing user behavior”. Workshop at Océ, July 15, 2010. Dorrestijn, S. (with P-P. Verbeek), “Product Impact: Redesign and Ethics”, Workshop at the DfU Symposium
- 151. 149Publications Product Impact, Enschede, November 11, 2010. Harkema, C.L.E. “Unconscious Uncertainty”. Workshop for all DfU project members, Utrecht, January 2011. Hoolhorst F.W.B., Anggreeni, I., Voort, M.C. van der. “The usability runway”. Workshop at the World Usability Day 2009, Delft, November 2009. Kim, C.J. and Harkema, C. ”MUST, A toolbox for usability”. Workshop at the World Usability Day 2009, Delft, November 2009. Kim, C.J. and Christiaans, H.H.C.M. “Unhappy users: What do designers wrong?” (subtitle: The impact of cognitive aspects in the usability of our household electronic products). UNIST in South Korea, June 2011 Kuijk, J.I. van. “Barriers and enablers for usability in practice, sharing results and getting feedback on a case study”. Workshop at the World Usability Day 2009, Delft, November 2009. Kuijk, J.I. van. “Usability in Practice”. Workshop at DfU project meeting, March 17, 2011, Utrecht, The Netherlands. Weevers, T.J.T. “Wayfinding in the variety of methods for user centred design”. Workshop at the Chi-Sparks conference, June 23, 2011, Arnhem, The Netherlands Websites www.designforusability.org Kuijk, J.I. van (2005 - 2011) weblog about consumer product usability: www.uselog.com.
- 152. 150 Design for Usability Methods Tools
- 153. 151 Design United, ‘platform for Dutch Research in Design’, is an initiative of the departments of Industrial Design of the three Technical Universities in the Netherlands. This initiative aims to increase the academic power of the field of Industrial Design and strengthen the innovative force of the Dutch industry. Industrial Design integrates knowledge from different disciplines and puts the user in a central position in the design process. It focuses both on the quality of the interface between user and product, and on the development of more complex systems incorporating multiple products and services. Within the context of increasingly complex societal issues, Industrial Designers have started to play a larger role in innovation processes. This requires knowledge, methodology, tools and new concepts concerning users, technology and business aspects. By addressing social issues and involving industry in a diversity of projects, design research is better attuned to the needs of existing and future companies. Two-way communication between universities and industry will also strongly contribute to the opening-up of existing and new knowledge and methodologies.
- 154. 152
- 156. Design for Usability was set up to integrate the worlds of research and product development, looking specifically at development issues in the rapidly changing field of product development of electronic goods. The researchers delved into the issues of user-centred development, working on case studies and interviewing users, designers, manufacturers - everyone involved in the development chain. An exciting, innovative project involving the three Dutch universities of technology, design companies and leading electronic product development manufacturers. The results of five years of hard work are presented here; the methods and tools that will help designers and practitioners design and develop better, more user-friendly products. This book reflects this practitioner-centred attitude. It takes a hands-on approach, provides in-depth discussion of the new methods and tools, how to apply them and what the benefits are. It is richly illustrated throughout and provides links to online resources. It is a must-read for any student, designer and product developer with a passion for user-centred design. www.designforusability.org ISBN 978-94-6186-077-4