User Centered Design: Interviews & Surveys.

INTERVIEWS & SURVEYS
Grau en Enginyeria Informàtica
User Centered Design

INTERVIEWS
Introducction
When?
What to consider?
Conducting an interview
Analysis and interpretation
Communicate the findings
Interviews & surveys. - User Centred Design 2 / 51

Introduction
• An interview is a guided conversation in which one part
seeks information from another.
• There are different types of interviews that can conduct,
depending on the constraints and requirements.
• They are flexible and can be used as a solo activity or
in conjunction with another user requirements activity
(e.g., following a card sort).

Introduction
• Goal:
• We will discuss
• when to use interviews,
• things to be aware of when conducting interviews,
• how to prepare for and conduct an interview, and
• how to analyze the data.
• Timeline:
• When should you conduct interviews?
• Preparing to conduct an interview
• Conducting an interview
• Data analysis and interpretation
• Communicate the findings.

When Should You Conduct Interviews?
• anytime in the UCD lifecycle
• when you want to obtain detailed information from individual users.
• excellent for innovation
• looking to develop a new product or service, interviews allow you to conduct an
outcomes analysis and retrieve the kind of detailed feedback from users
necessary for product innovation.
• help to prepare for another usability activity
• To prepare for a group task analysis
• identify the most frequent responses to questions in order to build a closed-
ended survey
• use the results of the interviews to design questions for a wants and needs
analysis
• following another usability activity to better understand the results
• e.g., why participants responded a certain way on your survey
• The end result of a set of interviews is an integration of perspectives
from multiple users

When DO NOT use an interview?
• if you are looking for information from a large sample of
the population.
• One-on-one interviews can take significant time to conduct more
resources than a survey
• For a large data set, surveys are a better option

Pros and cons of interviews
• Interviews are good for ...
• Collecting rich, detailed data
• Collecting information to design a survey or other usability activity
• Getting a holistic view of the system
• Interviews are not as good for ...
• Collecting data from large samples of people
• When you need to collect data very rapidly
• Collecting information on highly sensitive topics
• Be clear about your objectives and the kind of data you
are seeking and choose the activity that best suits your
needs

Things to be aware of
• Bias
• It is too easy to introduce bias into an interview !!!
• Do not encourage the participant to answer in a manner that does not reflect
the truth
• Do not influence a participant’s answers
• Honesty
social desirability, participants may
• provide a response that they believe is socially desirable or more acceptable
rather than the truth
Make it clear that you need to understand the way he or she actually works
Remind the participant that all information is kept confidential
prestige response bias, a participant may
• just agree to whatever the interviewer suggests in the belief that it is what the
interviewer wants to hear
• want to impress the interviewer and therefore provide answers that increase
his/her image
be neutral and encourage the participant to be completely honest with you
do not raise sensitive or highly personal topics

Things to be aware of
• People are innately honest !!
• It is an extremely rare participant who comes into your interview
with the intention of lying to you or not providing the details you
seek.
• BUT
• If you doubt the veracity of a participant’s responses, you can
always throw away the data and interview another participant

Preparing to Conduct an Interview
• selecting the type of interview to conduct
• wording the questions
• creating the materials
• training the interviewer
• inviting observers

Preparing for the interview session
When to complete Activity
As soon as possible • Meet with team to identify questions
• Meet with team to develop user profile
After identification of
questions and user Profile
• Create and distribute activity proposal
After the proposal has been
agreed to by all stakeholders
• Word questions appropriately and distribute to co-workers for review
After development of the
questions
• Identify and recruit users
• Assign roles for the activity (e.g., note-taker, interviewer)
• Prepare interview materials
• Acquire location
• Acquire incentives
• Prepare documentation (e.g., confidentiality agreement, consent form)
1 week before activity • Conduct pilot
• Make necessary changes to questions and procedure based on pilot
Day before interview • Call and confirm with participant(s)
• Remind stakeholders to come and observe the activity (if appropriate)
Day of interview • Set up location with all materials necessary
Interviews & surveys. - User Centred Design
A high-level timeline for you to follow when preparing for an interview
11 / 51

Identify the Objectives of the Study
• It is important for everyone (you and stakeholders) to
agree upon the purpose and objectives of the study
from the beginning
• Recommendation: included in your proposal to the stakeholders
and signed off by all parties
• Once agreement is reached, use the objectives of the
study as a guide to brainstorm questions

Select the Type of Interview
• Unstructured (or open-ended)
• the interviewer will begin with talking points but will allow the
participant to go into each point with as much or little detail as he/she
desires.
• questions (topics) are open-ended
• the interviewee is free to answer in any manner
• the topics do not have to be covered in any particular order
• Structured (controlled)
• closed-ended questions
• the interviewee must choose from the options provided.
• similar to conducting a survey verbally and is used by organizations
like the Census Bureau and Bureau of Labor Statistics.
• Semi-structured
• a combination of the structured and unstructured types
• They vary by the amount of control the interviewer places on
the conversation

Comparison of the three types of interview

Interviews via phone or in person?
• Sometimes it is more convenient for both participant and interviewer
to conduct the interviews via the phone and skip the travel time.
• Disadvantages to conducting phone interviews:
• participants on the telephone end the interviews before participants in face-to-
face interviews. It can be difficult to keep participants on the phone for more
than 20 minutes.
• Participants are more evasive and more hesitant to reveal sensitive
information about themselves.
• You cannot watch the participant’s body language, facial expressions, and
gestures, which can provide important additional information.
• Phones can be perceived as impersonal and it is more difficult to develop a
rapport with the participant and engage him/her over the phone.
• Web conferencing tools are available so that you may show artifacts to participants if
appropriate, but you still cannot gain the personal connection over the phone and
computer.

Writing the Questions
• Be BRIEF
• Short sentences, 20 words max. Break long sentences
• Wrong: “If you were waiting until the last minute to book a plane ticket to
save money and the only seats available were on the red-eye flight or had
two layovers (scales), what would you do?”
• Right: “If you were paying for the ticket on a four-hour airplane trip,
• Would you take a late-night/dawn-arrival flight that cost half as much? <answer>
• Would you accept a change of planes with a two-hour delay? <answer>
• What if you saved a quarter of the direct-flight fare? <answer>”
• Be CLEAR
• Avoid vague questions
• Avoid imprecise words like “rarely,” “sometimes,” “usually,” “few,” “some,”
and “most.”…
• Wrong: “Do you usually purchase plane tickets online?”
• Right: “How often do you purchase plane tickets online?”
The next step is to word the
questions so that they are clear,
understandable, and impartial
16 / 51

• Avoid BIAS
 leading questions
• Wrong: “Most of our users prefer the new look and feel of our site over
the old one. How do you feel?”
• Right: “How do you feel about the visual appearance of this website?”
 loaded questions (they typically provide a “reason” for a problem in
the question)
• Wrong: “The cost of airline tickets continues to go up to cover security
costs. Do you think you should have to pay more when you buy your
ticket, or should the government start paying more of the cost?”
• Right: “How much are you willing to pay for your plane ticket to cover
additional security at the airport?”

• Avoid BIAS
 interviewer prestige bias
• In this case, the interviewer informs the interviewee that an
authority figure feels one way or another about a topic and then
asks the participant how he or she feels
• Wrong: “Safety experts recommend using a travel agent instead of
booking your travel online. Do you feel safe using travel websites?”
• Right: “Do you feel that booking travel online is more or less
confidential than booking through a travel agent?”

• Avoid Predicting the future
• Rather than asking participants to predict the future or develop a
solution to a perceived problem, it is best to limit your questions to
desired outcomes
• Wrong: “What are your thoughts about a new feature that allows
you to instant message a travel agent with any questions as you
book your travel?”
• Right:
• “Would you like to correspond with a travel agent while you are booking
travel? <answer>
• What are some ways that you would like to correspond with a travel
agent while you are booking travel?”

• Inaccessible topics
• A participant may not have experience with exactly what you are asking about,
or may not have the factual knowledge you are seeking.
In these cases, be sure a participant feels comfortable saying that he or she does not
know the answer or does not have an opinion
Inform participants that there are no right or wrong answers
• Depending on memory
• EASY if the question seeks information about recent actions or highly
memorable events (e.g., your wedding, college graduation), it probably won’t
be too difficult
• DIFFICULT when people are asked about events that happened many years
ago and/or those that are not memorable
• Also avoid
• emotionally laden words: “racist”, “liberal”
• jargon, slang, abbreviations, and geek-speak (unless you are sure that your
user population is familiar with this terminology)
• Finally, take different cultures and languages into consideration

DO
• Keep questions under 20 words
• Address one issue at a time
• Word questions clearly
• Keep questions concrete and based on
the user’s experience
• Limit questions to memorable events or
ask participants to track their behavior
over time in a diary
• Provide memory aids like calendars to
help participants remember previous
events
• Use terms that are familiar to the user
• Use neutral terms and phrases
• Ask sensitive or personal questions
only if necessary
DON’T
• Force users to choose an option that
does not represent their real opinion
• Ask leading questions
• Ask loaded questions
• Base questions on a false premise
• Use authority figures to bias questions
• Ask users to predict the future
• Ask users to discuss unmemorable
events
• Use jargon, slang, abbreviations, geek-
speak
• Use emotionally laden words
• Use double negatives
• Ask sensitive or personal questions out
of curiosity
e
x
21 / 51

Test Your Questions
• 1rst with members of your team who have not worked on
the interview so far
• 2nd (optional) with a couple of actual participants

Players in Your Activity
• The participants
• General recommendation: to interview six to ten participants of each user
type, with the same set of questions
• BUT: There is no magic formula for determining the number of participants to
interview – the final answer is: “it depends.”
• The interviewer
• The task of the interviewer is to elicit responses from the participant
• How important it is for interviewers to be well-trained and experienced !!!,
without this, interviewers can unknowingly introduce bias into the questions
they ask
• Although we all hate to watch ourselves on video or listen to ourselves on a
tape recorder, it is helpful to watch/listen to yourself after an interview
• (optional) The note-taker
• a co-worker who is taking notes for you. You can focus more of your attention
on the interviewee’s body language and cues for following up
• (optional) The videographer
• Whenever possible, video record your interview session.
• It is NOT RECOMMENDED to have observers

Conducting an Interview
• The five phases of an interview
• Your role as the interviewer
• Monitoring the relationship with the interviewee
• DOs and DON’Ts

The five phases of an interview
Phase
Duration
min aprox
Procedure
1 5–10
Introduction (welcome participant,
complete forms, give instructions)
2 5–10
Warm-up (easy, non-threatening
questions)
3 85–100
Body of the session (detailed questions)
This will vary depending on the
number of questions.
4 5–10
Cooling-off (summarize interview, easy
questions)
5 5
Wrap-up (demonstrate that the interview is
now at a close. Distribute incentives, thank
participant, escort him/her out)
This should be
about 80% of
your interview
time with the
participant
25 / 51

Your Role as the Interviewer
• Do not interrupt
• Keep on track. Take care of transitions
between questions
• Silence is golden
• Remain attentive
• When needed, take a break at a logical
stopping point
• Asking the tough (difficult, embarrassing)
questions
• this is best done via surveys
• but if you think there is a need to ask a difficult
question in an interview
• wait until you have developed a rapport with the
participant.
• explain why you need the information
• Using examples
• Sometimes rewording the question is not
sufficient and an example is necessary for
clarification
• BUT, choose good examples !! (as we have
seen, some can introduce bias)
• Watch for generalities
• If you are looking for specific, detailed
answers, do not ask generalized questions
and ask for significant events.
• Do not force choices
• … when an interviewee has to make a choice
from a list of options and he/she says that it
does not matter
• Watch for markers
• key events to the participant that you can
probe into for more rich information
• Select the right types of probe
• “Can you tell me more about your decision?”
• Watch your body language
• Your tone and body language can affect the
way a participant perceives your questions
• Reflecting
• verify that you understand what the participant
has told you (summarize, reword, or reflect
the participant’s responses)
• Empathy and antagonism
• A skilled interviewer is able to empathize with
the participant without introducing bias

Monitoring the Relationship with the
Interviewee
• Watch the participant’s body language
• Fighting for control
• You are who controls the interview, not the interviewee
• Hold your opinions

Data Analysis and Interpretation
• Do it soon: the longer you wait to get to the analysis, the
less you will remember about the session
• debrief session with your note-taker and any other observers to
discuss what you learned
• review the recording and add any additional notes or quotes
• If stakeholders ask you for results before a study is
complete.
• Just give them just a few interesting quotes that stand out in your
mind
28 / 51

• Categorizing
• For example, how many people so far have selected each option in
a multiple-choice question, or
• what is the average rating given in a Likert scale question?
Mean Graph of frequencies
29 / 51

Affinity diagram
Qualitative Analysis Tools
• http://www.qualisresearch.com/default.htm
• http://www.qualrus.com
• http://www.qsrinternational.com/products.aspx
• SPSS Software: http://www-01.ibm.com/software/analytics/spss
30 / 51

Communicate the Findings
• A good report illuminates all the relevant data, provides a
coherent story, and tells the stakeholders what to do next
• Group results by:
• Topic
• Participant
• You can use:
• A poster
• “Personas” and/or “Scenarios”
• Identify follow-up activities
(e.g., a survey)
• Table of recommendations
Recommendations
• Verbal Presentation is Essential
• Start with the good stuff
• Use visuals
• Prioritize and start at the top
• Avoid discussion of implementation
or acceptance
• Avoid jargon
read Chapter 14, Concluding Your Activity
31 / 51

Start preparing your interview
• Identify the objectives
• Decide the type of interview
• Write the questions
• From the project do:
• Write the whole process of an interview
• Introduction
• Questions
• Analysis of results
• Communication of results
• RECOMMENDATION: once drafted the interview, test it
with someone other than group mates
e
x
32 / 51

SURVEYS
Introduction
When?
What to be aware of
Creating and Distributing your survey
Data Analysis and interpretation
Communicate the findings

Introduction
• Extremely effective to gather information about users.
• Surveys are a way to reach a larger number of people
than the other methods typically allow
• Surveys are perceived as very easy to create, BUT the
problem is that a valid and reliable survey can be very
difficult to design.
• It’s just bunch of questions, but
• which are these questions???
• How to distribute???
• A poorly designed survey can provide meaningless or
inaccurate information.

When Should We Use a Survey?
• For a new product (or a new version), a survey can be a
great way to start your user requirements gathering
For a new product
• Help you identify your
potential user population
• Find out what they want and
need in the product you are
proposing
• Find out at a high level how
they are currently
accomplishing their tasks.
For an existing product
• Learn about the user
population and their
characteristics
• Find out users’ likes and
dislikes about the current
product
• Learn how users currently
use the system

What To Be Aware of
• Response Bias
• Survey = self-report !!
• Sometimes respondents may answer questions based on how they
think they should be answered, rather than truly expressing their own
opinions (social desirability)
• Try to provide your respondents with complete anonymity
It tends to be more of a problem in interviews rather
than surveys, because surveys are usually anonymous
whereas in an interview the interviewee must answer
directly to the interviewer.
However, it is still a factor that one must be aware
of in surveys.
36 / 51

What To Be Aware of
• Response Rate
• Unfortunately, reality of surveys is that not everyone is going to respond
• Estimation between 20% and 60%
• Some things you can do to improve the response rate:
• Include a personalized information about the purpose of the study and how
long it will take.
• Reduce the number of open-ended questions.
• Keep it short.
• Make the survey attractive and easy to comprehend.
• Make the survey as easy to complete and return as possible.
• For example, if you are sending surveys in the mail, include a self-addressed envelope
with pre-paid postage.
• Follow up with polite reminders
• Consider offering a small incentive for their time
• Contacting non-respondents in multiple modes has been shown to
improve response rates

Creating and Distributing the Survey
• Preparation
timeline

• Identify the Objectives of Your Study
• Do not just jump in and start writing your survey. You need to do
some prep work
• Ask yourself
• Who is the survey for?
• What information are you looking for?
• How will you distribute the survey and collect responses?
• How will you analyze the data?
• Who will be involved in the process?
• Players in Your Activity
• (see interviews)

• Composing the Questions
• Keep It Short
• Start with brainstorming of all the potential questions that you might like
to ask
• Then, reduce the initial question set
• (remember) Be aware about asking Sensitive Questions
• Question Format and Wording
• The questions and how they are presented are key in accomplishing
this
• Response format
• Open-ended questions allow the users to compose their own responses
• Closed-ended questions require participants to answer the questions by
either
• Providing a single value or fact
• Selecting all the values that apply to them from a given list
• Providing an opinion on a scale.

open-ended questions
• PROS
• they are much easier to create
• CONS
• make data analysis tedious and complex
• responses can be difficult to comprehend
• and you typically will not have an opportunity to follow up and clarify
with the person who responded
• can make the survey longer to complete
• thereby decreasing the return rate
open-ended
questions are best
reserved for
interviews
41 / 51

Multiple-choice questions
• Multiple-response
• Participants can choose more
than one response from a list
of options.
• Single-response
• Participants are provided with
a set of options from which to
choose only one answer.
• Binary
• As the name implies, the
respondent must select from
only two options.
“What types of travel have you booked
online? Please select all that apply.”
 Airline tickets
 Train tickets
 Bus tickets
 Car rental
 None of the above
“How often do you book travel online?”
 Once a month
 4–6 times per year
 1–3 times per year
 I never book travel online
 Yes
 No
True
False
Agree
Disagree
42 / 51

Rating scales
• The Likert scale is the most frequently used rating scale
• Scale question asks users to give a priority rating for a range of options
• As you are not comparing the options given against one another, more than one
option can have the same rating
• Ranking scales, the same as scale questions but the respondent is
allowed to use each rank only once
“Rate the importance of each of the following features in a
travel site from 1 to 4, with 1 being not at all important and 4
being very important.”
___ Low prices
___ Vast selection
___ Chat and phone access to a live travel agent
___ A rewards program for purchasese
43 / 51

DO
• Keep questions under 20 words
• Address one issue at a time
• Word questions clearly
• Provide precise options in closed-ended
questions
• Equally space the range of options in closed-
ended questions
• Ask users to discuss desired outcomes
• Provide background information only if
necessary and keep that information factual
• Keep questions concrete and based on the
user’s experience
• Limit questions to memorable events or ask
participants to track their behavior over
• time in a diary
• Provide memory aids like calendars to help
participants remember previous events
• Use terms that are familiar to the user
• Use neutral terms and phrases
• Ask sensitive or personal questions only if
necessary
DON’T
• Force users to choose an option that
does not represent their real opinion
• Ask leading questions
• Ask loaded questions
• Base questions on a false premise
• Use authority figures to bias answers
• Ask users to predict the future
• Ask users to create solutions
• Ask users to discuss unmemorable
events
• Use jargon, slang, abbreviations, geek-
speak
• Use emotionally laden words
• Use double negatives
• Ask sensitive or personal questions out
of curiosity
e
x
Question wording
45 / 51

Question wording
• Avoiding Pitfalls
• avoid vague options like “few,” “many,” and “often.”
• When providing a range of answers to choose from
• Keep ranges equal in size and never allow the ranges to overlap (e.g.,
“0–4,” “5–10,” “11–15”)
• consider providing an “out” option to respondents on each question
• “None of the above”, “No opinion”, “Not applicable”
• “Other”
• In this case provide a blank space for the participant to insert his/her own
answer
• This changes the question from closed-ended to open-ended
• Free-response answers will have to be analyzed by hand

Standard Items to Include
• Title
• Keep it short and sweet
• Instructions
• Contact information
• Purpose
• Time to complete
• Confidentiality
• The person’s identity will not be associated in any way with the data
provided
• You should make a clear statement of confidentiality at the beginning
of your survey
• Anonymity
• Even you, the researcher, cannot associate a completed survey with
the respondent’s identity

• Initial Assessment
• Types of Calculation
• Mean
• Median
• Mode
• Measures of dispersion
• Frequency
• Measures of association
• Complex statistics
Measures of Association
Graph comparing
people who book
hotels and cars
on the web versus
those who do not
48 / 51

Communicate the Findings
• Same recommendations as interviews

Study case
• Using the project, DO:
• Write a survey
• Introduction
• Questions (Minimum 25 questions)
• Analysis of results
• Communication of results
• Recommendation: test it with some people out of your
group colleagues
e
x
50 / 51

More information
• http://boxesandarrows.com/a-stakeholder-interview-
checklist
• http://boxesandarrows.com/the-general-stakeholder-
interview
• http://alistapart.com/article/interviewing-humans

User Centered Design: Interviews & Surveys.

More Related Content

What's hot (20)

Similar to User Centered Design: Interviews & Surveys. (20)

More from DCU_MPIUA (20)

Recently uploaded (20)

User Centered Design: Interviews & Surveys.

Editor's Notes