SlideShare a Scribd company logo
Preference and Desirability Testing: Measuring Emotional Response to Guide DesignMichael HawleyChief Design Officer, Mad*PowPaul DoncasterSenior User Experience Designer, Thomson Reuters
AgendaWhy we should careWhy it’s not always as simple as asking:“Which option do you prefer?”Methods to considerCase Study: Greenwich HospitalCase Study: WestlawNextSummary/Comparison
Why we should care3
Impressions Count
An important role of visual design is to lead users through the hierarchy of a design as we intendFor interactive applications, a sense of organization can affect perceived usability and, ultimately, users’ overall satisfaction with the productFunctional Desirability
Visceral Emotions
7Fogg’s Behavior ModelCore motivators include:Pleasure/pain
Hope/fear
Acceptance/rejectionhttp://www.behaviormodel.org/
8Positioning Desirability Studieshttp://www.xdstrategy.com/2008/10/28/desirability_studies/
“Which do you prefer?”9
Quantity, granularity breed apathy
Poor articulation“It reminds me of…”“It’s nice and clean.”“There’s just something about it . . .”“I ordinarily don’t like red, but for some reason it works here . . .”“It’s better than the other ones.”
What Stakeholders Should Care About“We should go with design C over A and B, because I feel it evokes the right kind of emotional response in our audience that is closer to our most important brand attributes.”
Methods to Consider13
Present three different concepts or ideas to participants, and ask them to identify how two of them are different from the third and why.14Triading
Broad, experience-based questionnaires, that also include questions relating to visual appeal and aestheticsSUS (System Usability Scale)
QUIS (Questionnaire for User Interface Satisfaction)
WAMMI (Website Analysis and Measurement Inventory)15Qualitative Questionnaires
Show participants a user interface for a very brief moment, then take it away. Participants recall their first impression, then moderator probes for meaning.Helpful for layout decisions, prominence of content, labels
www.fivesecondtest.com16Attention designers: You have 50 milliseconds to make a good first impressionQuick Exposure Memory Tests
Electroencephalography (EEG): Brain activity
Electromyography (EMG): Muscles and ExcitementElectrodermal Activity (EDA): Sweat, Excitement
Blood Volume Pressure (BVP): Arousal
Pupil Dilation: Arousal and Mental Workload
Respiration: Negative Valence or Arousal17Physiological and Neurological
18PrEmo Emotional MeasurementDr. Pieter Desmet, Technical University of Delfthttp://www.premo-online.com
19http://www.microsoft.com/usability/uepostings/desirabilitytoolkit.docProduct Reaction Cards
Case Study: Greenwich Hospital20
Determine intended brand attributes (and their opposites)21Product Reaction Cards: Before You BeginLeverage existing marketing/brand materialsAlternatively, stakeholder brainstorm to identify key brand attributes/descriptors using full list of product reaction cards as a startTip: “If the brand was a person, how would it speak to your customers?”
MethodologyInclude 60/40 split of positive and negative wordsTarget 60 words, optimized to test brandSimple question: “Which of the following words do you feel best describe the site/design/product (please select 5):”One comp per participant, or multiple comps per participant (no more than 3)ParticipantsQualitative: Paired with usability testingQuantitative: Target minimum of 30 per option if possible22Product Reaction Cards: Conducting
23Process - AnalyzingCalculate percentage of positive and negative attributes per designVisualize overall sentiment of feedback using “word clouds” (see wordle.net)68% Positive32% Negative
Align the website with the character of the Hospital
Update the site after nearly 10 years
Counter impressions that Greenwich is more than just maternity and elder care
Communicate that they are long-standing members of the community24Case Study: Greenwich Hospital Website Redesign
25Case Study: Greenwich Hospital Website Redesign 3 visually designed comps
50 people reacted to each comp (quantitative) via survey
Additional feedback obtained via participant interviews (qualitative)Survey QuestionsHello, I am requesting feedback on a website I am working on.  Your answers let me know if the site is conveying the right feel. 1. What are your initial reactions to the web site? 2. Which of the following words best do you feel best describe the site (select 5):
26Three Different Visual Designs
27Results: Concept 112% Negative88% Positive
28Results: Concept 287% Positive13% Negative
29Results: Concept 35% Negative95% Positive
Mix of qualitative and quantitative is key
Qualitative helps provide color to the results
Quantitative resonates with stakeholders and executives
Position results as one form of input to decision-making process, not declaring a “winner”
Simple, cost-efficient way to assess audience’s emotional response to a design30Lessons Learned
Case Study: WestlawNextUX Research Team:Drew DrentlawShannon O’BrienBill QuieNovember Samnee31
Desirability and Preference Testing - UPA International 2011
for Phase 1 Use large sample sizes to establish a design “baseline,” from which to advance the design direction in subsequent iterations
Isolate preference trends for specific page design aspects
Determine tolerance for manipulation of the site “brand”
Maintain tight securityGoals
Sessions were held in 4 cities over 5 daysSeattleDenverMemphisMinneapolis-St. Paul4 sessions were held per day, with a maximum of 25 participants per session1.5 hours allotted per study, most participants finished in less than 1 hour319 participants successfully completed their sessionsPhase 1: Logistics & Execution
Participants completed the study at individual workstations at their own paceAll workstations included a 20” monitor, at 1024x768 resolutionPhase 1: Logistics & ExecutionMemphis, TN,  May 2009
Brief review of Westlaw critical screens Positive/negative word selection to describe Westlaw 36Positive/negative product descriptors
Each set of Element variations were viewed in full screenParticipant selects “top choice” by dragging a thumbnail image to a drop area37Homepage: Design Elements
38
Homepage: Design Elements (1)All options viewed in full screenParticipant selects “top choice” by dragging a thumbnail image to a drop area
Desirability and Preference Testing - UPA International 2011
Desirability and Preference Testing - UPA International 2011
Visual Weight (6 options)Use of Imagery (8 options)Components (4 options)Search Area (4 options)Palette (10 options)Homepage: Design Elements
19 HP designs viewed in full screen (randomized)All 19 options are presented again; participant assigns a rating using a 10-point slider.Top 5 and Bottom 2 choices are positioned in order of rating values on one long, scrollable page. Next to each design displayed, rates key aspects for each design on a 5-point scaleHomepage: Design Gallery
Desirability and Preference Testing - UPA International 2011
Desirability and Preference Testing - UPA International 2011
Desirability and Preference Testing - UPA International 2011
Desirability and Preference Testing - UPA International 2011
Desirability and Preference Testing - UPA International 2011
Repeat the process for Results List design:New Results ListDesign Elements
Column Collapsing (4 options)
Column Separation (2 options)
Theme/Color (8 options)
Design Gallery
14 Results Lists designs (randomized)
Key Aspects Rated
Color scheme
Global Header

More Related Content

PPTX
Preference and Desirability Testing: Measuring Emotional Response to Guide De...
DOCX
Unit3 productdevelopmentconcepttopf
PDF
Etm551 lecture05
PPTX
Basic Engineering Design (Part 4): Selecting the Best Solution
PPTX
Basic Engineering Design (Part 3): Designing Solutions
PPTX
Usability methods to improve EMRs
PPTX
Basic Engineering Design (Part 5): Constructing a Prototype
PPTX
Basic Engineering Design (Part 2): Researching the Need
Preference and Desirability Testing: Measuring Emotional Response to Guide De...
Unit3 productdevelopmentconcepttopf
Etm551 lecture05
Basic Engineering Design (Part 4): Selecting the Best Solution
Basic Engineering Design (Part 3): Designing Solutions
Usability methods to improve EMRs
Basic Engineering Design (Part 5): Constructing a Prototype
Basic Engineering Design (Part 2): Researching the Need

What's hot (18)

PPTX
Basic Engineering Design (Part 7): Presenting the Solution
PPT
UXPA International 2013 The Note-Taker's Perspective
PPTX
Initiating Creative Design by Brainstorming
PPTX
Basic Engineering Design (Part 1): Identify the Problem
PPTX
Basic Engineering Design (Part 6): Test and Evaluate
PDF
Design thinking
PPSX
Usability Testing Fundamentals
PDF
Usability testing
PPTX
Modelling guidance 2014
PDF
Research Proposal B06 Cognitive Conflicts 4269861
PDF
Design methods for emotions
PPTX
Basic Engineering Design (Part 8)): Redesigning & Iterating
PDF
Kevin karty advances in quant - 2011
PPTX
Study notes me-112-concepts-in-engineering-design-unit-1
PDF
Chapter2 framework-for-design
PDF
UI/UX foundations - Intro to Design
PDF
UI/UX Foundations Part 1 - Design
PDF
Usability testing / Nearly everything you need to know to get started
Basic Engineering Design (Part 7): Presenting the Solution
UXPA International 2013 The Note-Taker's Perspective
Initiating Creative Design by Brainstorming
Basic Engineering Design (Part 1): Identify the Problem
Basic Engineering Design (Part 6): Test and Evaluate
Design thinking
Usability Testing Fundamentals
Usability testing
Modelling guidance 2014
Research Proposal B06 Cognitive Conflicts 4269861
Design methods for emotions
Basic Engineering Design (Part 8)): Redesigning & Iterating
Kevin karty advances in quant - 2011
Study notes me-112-concepts-in-engineering-design-unit-1
Chapter2 framework-for-design
UI/UX foundations - Intro to Design
UI/UX Foundations Part 1 - Design
Usability testing / Nearly everything you need to know to get started
Ad

Similar to Desirability and Preference Testing - UPA International 2011 (20)

PPTX
Preference and Desirability Testing, Measuring Emotional Response to Guide De...
DOCX
A3 Problem Solving Template v1.2 (April 2015) by Henrik Knibe.docx
DOCX
Organic vs Mechanistic StructuresMechanistic OrganicIn.docx
PPT
Text, Tags and Thumbnails: Latest Trends in Bioscience Literature Search
PPT
MY DESIGN PROVCESS.ppt
PPTX
趨勢科技案例分享 - 與專家一起共舞 Design Sprint
PDF
Purpose Before Action: Why You Need a Design Language System
PDF
Design Thinking by Simon Barna
PPTX
Environmental building design performance modelling and simulation
PPTX
Market profiles report example
PDF
G1_Product-Design-G1_Product-Design-G1_PD
PPT
Ppp Of Simulation Development2
PPT
Chapter 9 id2e_slides
PDF
Basics of-usability-testing
PPT
Introduction to Prototyping - Scottish UPA - June 2011
PPT
Mob4hire user feedback techniques for SEE 2009
PPTX
Managing Stakeholders (LS2010, Session 804)
PDF
UXSG2014 Workshop (Day 1) - Collaborative Experience design workshop (Hewlett...
PPTX
What you can learn from usability testing
Preference and Desirability Testing, Measuring Emotional Response to Guide De...
A3 Problem Solving Template v1.2 (April 2015) by Henrik Knibe.docx
Organic vs Mechanistic StructuresMechanistic OrganicIn.docx
Text, Tags and Thumbnails: Latest Trends in Bioscience Literature Search
MY DESIGN PROVCESS.ppt
趨勢科技案例分享 - 與專家一起共舞 Design Sprint
Purpose Before Action: Why You Need a Design Language System
Design Thinking by Simon Barna
Environmental building design performance modelling and simulation
Market profiles report example
G1_Product-Design-G1_Product-Design-G1_PD
Ppp Of Simulation Development2
Chapter 9 id2e_slides
Basics of-usability-testing
Introduction to Prototyping - Scottish UPA - June 2011
Mob4hire user feedback techniques for SEE 2009
Managing Stakeholders (LS2010, Session 804)
UXSG2014 Workshop (Day 1) - Collaborative Experience design workshop (Hewlett...
What you can learn from usability testing
Ad

More from hawleymichael (6)

PDF
Research and Design Methods - Healthcare Experience Design Conference 2012
PPTX
Educating, Mentoring and Motivating a UX Team
PPTX
Beyond Usability Testing: Assessing the Usefulness of Your Design
PDF
M Hawley Desirability Studies Boston Upa Presentation V4
PDF
Guided Selling - An Introduction and User Study Results
PDF
Extending Card Sorting
Research and Design Methods - Healthcare Experience Design Conference 2012
Educating, Mentoring and Motivating a UX Team
Beyond Usability Testing: Assessing the Usefulness of Your Design
M Hawley Desirability Studies Boston Upa Presentation V4
Guided Selling - An Introduction and User Study Results
Extending Card Sorting

Recently uploaded (20)

PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
Review of recent advances in non-invasive hemoglobin estimation
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
Approach and Philosophy of On baking technology
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
NewMind AI Monthly Chronicles - July 2025
PDF
Machine learning based COVID-19 study performance prediction
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PPTX
Cloud computing and distributed systems.
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Network Security Unit 5.pdf for BCA BBA.
PDF
Spectral efficient network and resource selection model in 5G networks
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Chapter 3 Spatial Domain Image Processing.pdf
“AI and Expert System Decision Support & Business Intelligence Systems”
Review of recent advances in non-invasive hemoglobin estimation
The AUB Centre for AI in Media Proposal.docx
Approach and Philosophy of On baking technology
Unlocking AI with Model Context Protocol (MCP)
Per capita expenditure prediction using model stacking based on satellite ima...
Digital-Transformation-Roadmap-for-Companies.pptx
20250228 LYD VKU AI Blended-Learning.pptx
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
NewMind AI Monthly Chronicles - July 2025
Machine learning based COVID-19 study performance prediction
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Cloud computing and distributed systems.
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Network Security Unit 5.pdf for BCA BBA.
Spectral efficient network and resource selection model in 5G networks

Desirability and Preference Testing - UPA International 2011

Editor's Notes

  • #5: Impacts a product's or application's perceived:UtilityUsabilityCredibility
  • #7: If users have a positive impression of the design aesthetics, they are more likely to overlook or forgive poor usability or limited functionality. With a negative impression, users are more likely to find fault with an interaction, even if a product’s overall usability is good and the product offers real value.
  • #8: High desirability feeds into the motivational factors that help trigger target behavior.
  • #9: High desirability feeds into the motivational factors that help trigger target behavior.
  • #11: The simplicity of the question doesn’t work well with larger numbers of design options, especially if some are highly similar
  • #12: People can have a difficult time articulating what it is about a design they like or dislike
  • #13: The whys are important for stakeholder acceptance (branding guidelines)Business sponsors and stakeholders often want substantial customer feedback to assure them a given direction is correct.
  • #15: TriadingQualitative interview technique that reveals constructsElicits attributes that are important to users in their vocabularyResearcher asks the participant to identify how two of the three examples are different from the thirdIn typical user research interviews, a researcher asks participants about their thoughts on a defined list of topics. The disadvantage of this approach is that the researcher may be inquiring about topics that are of little value or significance to the experience of the participants. Generally, participants will dutifully answer questions about any topics we ask them about, without thinking more broadly, going beyond the limits our questions impose, or interrupting us to tell us about dimensions that may be more relevant to them. Participants assume researchers are interested in studying the particular topics they’ve included in their interview scripts and don’t raise other issues that might be more pertinent to their overall experience with a product or potential design.Triading is a method that allows a researcher to uncover dimensions of a design space that are pertinent to its target audience. In triading, researchers present three different concepts or ideas to participants and ask them to identify how two of them are different from the third. Participants describe, in their own terms, the dimensions or attributes that differentiate the concepts. Participants follow this process iteratively—identifying additional attributes they feel distinguish two of the concepts from the third until they can’t think of any other distinguishing factors. By repeating this process across multiple participants, researchers can see trends that define audience segments or personas.The benefit of this process is that it uncovers dimensions of a particular domain that are important to the target audience rather than the researcher or designer. In addition, the dimensions participants identify are generally emotional aspects that it is important for experience designers to consider. For example, participants may describe differences in groups as “warm” versus cold” or business-like” versus fun.” Designers can then use the most relevant or common dimensions as inspiration for further design and exploration.
  • #16: Benefits – straightforward and easy to administer on a large scaleNegatives – if you want to pick more than a clear winner but rather understand the emotional connections/reactions to each design this will not lend itself to that.
  • #17: Obvious examples are consumer electronics or other retail productsAlso appropriate for applications in healthcare, insurance, financial, travel, etc.
  • #18: Sensors track participants’ physiological measurements to particular designs. Changes in suggest a particular emotional response.Paired with attitudinal and self-reporting surveys measurements give a multifaceted view of emotional reactions to a design
  • #19: Respondents are being asked: "To which extent do the feelings expressed by the characters correspond with your own feelings towards the stimulus?“Building on the responses of many people allows you to abstract valuable data pertaining to the emotional performance of your website, product, service.
  • #25: “luxurious, approachable, friendly, capable, multi-cultural/inclusive, established”
  • #28: “My initial reaction to this web site is that it seems kind of plain. There is not much going on in the page, and the colors seem kind of drab.”“This is a nice looking website. It is well designed, well laid out, and is appealing to look at. It makes me want to continue to navigate the site to learn more. “
  • #29: “Men don’t really go with children… where there’s a baby, there must be a mother.”“My initial reaction to the website is that it seems very clean and modern. I like the layout, it looks like its easy to find information.”
  • #30: “I felt love. I saw a mother holding a child…that’s pretty touchy. The site looks good, and it makes the hospital trustworthy.”“My initial reaction was that the hospital is represented by a caring, warm and friendly website.”
  • #57: *** review no big fonts **
  • #71: Evolution of how the document separates from the header and tools, and related topics on right
  • #74: In order of ascending practicality
  • #76: In order of ascending practicality