Rapid Prototyping and Usability Testing - HUXPA

Rapid
Prototyping and
Usability Testing
Houston UXPA
October 2016

© Pariveda Solutions. Confidential & Proprietary.2
Our Clients
Pariveda solves the complex problems of
clients ranging from Fortune 100 to Global
2000 to startup companies and spanning
multiple industries.
Clients partner with us for our high-caliber
combination of technology and business
problem-solving experts, our high-quality
delivery consistency and our focus on
building lifetime relationships. We have
served over 400 clients since inception and
currently have projects with approximately
225 of those clients.
Key Details
Our Locations
Pariveda Solutions Inc. is a leading management consulting firm delivering strategic services and
technology solutions. Our focus is simple. Start with the right people, deliver consistent value and
partner enthusiastically with our clients. We grow and deploy talented people to solve technical
and strategic challenges. We are passionate about delivering exceptional value to our clients.
Our SolutionsPariveda Overview
Strategy
Mobility
Cloud
Data
Portals & Collaboration
CRM
Custom Software
Enterprise Integration
User Experience

Why don’t people adopt the
changes we design to benefit them?
What will help us match our vision to
user needs?
How can I collect early unbiased
feedback on my project?

PROBLEM:
Why don’t people adopt the changes we
design to benefit them?

1st Why
• People don’t adopt a change when they feel the
effort to do so is greater than the value they will get
from it (EFFORT > VALUE)
• Team didn’t fully understand the needs and pain
points they were trying to solve
• Main benefit wasn’t aimed at users most impacted
Change agents don’t match the vision to
user needs

2nd Why
• Validation of value proposition needs to occur with
the people actually impacted, not HiPPOs
• Feedback must be received before the course is
completely set
They don’t get quality feedback early
enough, or from the right people

3rd Why
• Similar alternatives:
• They are not used to putting ideas in front of end-users
• It may be difficult to communicate with the right
people for some reason
• All boil down to not prioritizing this validation
They feel confident in their solution, or
don’t realize their biases are causing them
to miss important flaws

4th Why
• Collecting feedback requires pausing measurable progress
• In justifying the pause to direct managers/leaders they have
to admit there are unknowns
• Matching to user expectations can be messy
They feel uncomfortable acknowledging that
they don’t have all of the answers, or don’t
want to be proven wrong

5th Why
• In a “fixed-mindset” environment participants feel required
to protect themselves, the risk of losing face outweighs the
benefit of open collaboration, especially when they feel
they are on the right track
• Teams optimize process for showing progress towards short
term milestones, not necessarily for gaining understanding
that will lead to the ultimate success of the effort
They feel pressure to produce contributions
and/or to demonstrate expertise

When users fail to adopt, it is a function of
1. Team bias towards believing they know
what end users will want
2. An environment that requires teams to
“protect” themselves by showing
constant short-term progress
As much as or more so than it being a reflection of design skills
or other factors.
So what does this mean?

What will help us match our vision
to User Needs?
F E E D B A C K
U N B I A S E D
E A R LY

Before Building
learn what will work
• Challenge ideas while they are still
malleable
• Start as soon as you can get
something tangible in front of
people
• An early start allows time to
adjust, lowers re-work
EARLY

Unbiased Users
reveal Unknown/Unmet
• Vision familiarity blinds you to
flaws that slow adoption
• Engage with the people who will
be actually impacted
• Customers feel invested, become
champions of the new state
UNBIASED

Confirmation Bias
 Choice
supportive Bias
 Congruence Bias
 Bandwagon Effect
Blind Spots
 Availability
Heuristic
 Ostrich Effect
 Observer-
Expectancy Effect
Curse of Knowledge
 IKEA Effect
 Mere Exposure
Effect
 Empathy Gap
Cognitive biases prevent us from understanding the
effort vs. value equation for end users
Graphic source: http://blog.elation.com/coaching-blog/cognitive-biases-in-leadership

Test Ideas
to find the best
• Build loops that feed back
meaningful information
• Uncover what is confusing and
what people don’t need
• Demonstrate progress with
experimental validation
FEEDBACK

How can I collect
F E E D B A C K
U N B I A S E D
E A R LY
on my project?

 Strategic Process Improvements
 Product Development (Google Design
Sprint)
 Organization Design
 Personal/Career Development
(Deliberate Practice)
 Business Intelligence
 Software Development
Early Unbiased Feedback is a mindset that applies in
many project situations

User experience design is the process of involving
users throughout the project
Discover
1) Product Vision
2) Contextual
Inquiries
3) User Stories
4) Personas
Design
Develop
5) Process Flows
6) Sitemap/IA
7) Wireframes
8) Style guide
9) Prototype
10) User Testing
11) Review
12) Approval
ITERATE

Adjust your process to fit
the use case and your
client’s reality
Adjust
Depending on the
project you might seek
feedback every 2 weeks,
or after defined project
milestones
Schedule
Start from the artifacts
created to cement
internal team
understanding or for
business stakeholders
Artifacts
Whenever you create
wireframes/prototypes,
you should get them in
front of real people
Test
Start early and repeat as frequently as you can
create something tangible

 Graph paper wireframes are easy to
create and can be tested for feedback
 Have testers walk through 10-15 “tasks”
and see what they are able to
complete
 Tasks should be high level, such as
“Check the status of your order” NOT
“Click the start button”
 Have a proctor act as “the computer”
moving them between screens and
explaining what happened
 Sample usability testing template:
https://parivedasolutions.box.com/v/u
sability-testing-template
 Have the tester talk out loud, and ask
questions to create a dialog
 Follow up with standard UX survey,
such as the SUS
With Paper Wireframes

A few sessions of feedback will help
you confirm:
 Layout
 Navigation
 Terminology
 Discoverability
 Learnability
 Functionality
User testing is invaluable for evaluating if app
features meet users’ needs

Usability Test + System Usability Scale

 The System Usability Scale is not diagnostic and will not tell you what specific
problems you face, but it will give you a red or green light to know how badly
your usability needs work.
 The average System Usability Scale score is 68. If your score is under 68, then
there are probably serious problems with your website usability which you should
address. If your score is above 68, then you can relax a little bit.
 Here’s an overview of how your scores should measure:
 80.3 or higher is an A. People love your site and will recommend it to their friends
 68 or thereabouts gets you a C. You’re doing OK but could improve
 51 or under gets you a big fat F. Make usability your priority now and fix this fast.
Scoring the SUS
http://uxpajournal.org/wp-content/uploads/pdf/JUS_Brooke_February_2013.pdf

SUS Scoring Example
Question User 1 User 2 User 3 User 4 User 5 User 6 User 7 User 8 User 9 User 10 User 11 User 12 User 13 User 14 User 15 User 16 User 17
1. I think that I would like to use
the app frequently.
5 4 5 5 5 4 5 5 4 5 4 4 5 5 5 4 5
2. I found the app unnecessarily
complex.
1 3 2 1 5 2 2 2 2 2 2 3 2 2 1 2 1
3. I thought the app was easy to
use.
5 3 4 4 5 4 5 4 4 5 4 3 4 4 5 4 4
4. I think that I would need the
support of a technical person to
be able to use the app.
2 4 1 1 1 2 2 2 1 1 2 2 2 2 1 1 2
5. I found the various functions in
the app were well integrated.
5 4 5 4 4 4 5 4 5 4 4 4 4 5 5 5 5
6. I thought there was too much
inconsistency in the app.
1 3 1 1 2 2 2 3 2 2 2 3 3 2 1 2 2
7. I would imagine that most
people would learn to use the app
very quickly.
5 3 5 5 3 5 5 3 5 4 4 4 4 4 4 3 3
8. I found the app very
cumbersome to use.
2 3 4 2 2 2 2 2 2 1 2 3 2 2 2 3 2
9. I felt very confident using the
app.
1 3 4 4 3 4 5 3 4 5 3 3 3 4 4 4 4
10. I needed to learn a lot of
things before I could get going
with the app.
2 4 1 2 1 2 2 2 2 1 2 2 1 2 2 1 1
SUS score 82.5 50 85 87.5 72.5 77.5 87.5 70 82.5 90 72.5 62.5 75 80 90 77.5 82.5
Cumulative Score:
78
Letter Grade:
B
Sample Size:
17
COMMENTS:
User 2 (SUS Score 50): I thought the system was generally easy to use but there was some difficulty finding certain issues in the sytem. I think its getting comfortable with this system and figuring out what needs to be done wil make thing
User 3 (SUS Score 85): After getting used to the new interface it was easier to navigate. After using both workfront and salesforce the programe integrated both program strengths
User 4 (SUS Score 87.5): I think once we get started on this program we are all really going to like it. It's easy to navigate and having everything all in one place will make our process run much smoother!
User 5 (SUS Score 72.5): I believe the system was extremely easy to use once getting used to it. The only thing I did not see was where the project goes after you are complete with your task. For anyone new they will not see the process
User 8 (SUS Score 70): It would be nice to have multiple windows open if needed.
User 8 (SUS Score 70): I usually have my tasks always open, and then 1 or 2 jobs I'm working on.
User 8 (SUS Score 70): When searching for issues, or projects that have issues, I find it helpful to sort issues by the type or name. Its easy to get in a groove when there s many of 1 or 2 types of issues
User 9 (SUS Score 82.5): Only because it was the first time ever seeing the program with short training. I think everyone would pick it up quickly once you knew where to click. It was very intuitive
User 9 (SUS Score 82.5): Having a general "overview" chat section would be easier to see all updates vs. having "task" chats
User 11 (SUS Score 72.5): With consistent use I would feel comfortable using the software
User 16 (SUS Score 77.5): Need Tiers for payment (winback)

 Can use same testing procedure as
with paper wireframes
 Many UX prototyping tools available
(Axure, Invision, Balsamiq, etc)
 If you have a target technology
platform / development team
identified, we have found it works
best to simply wire together front end
screens
 Just as quick
 More true to life
 Better mold for developers to follow
 Example:
http://derrickbowen.com/fsm-
mockup/
With a Clickable Prototype

During End User Testing pre and post Go-Live
 Include as part of standard “User
Acceptance Testing” process
 Can use same testing procedure as
with paper wireframes & prototypes
 SUS score provides a standard
benchmark to gauge user satisfaction
over project life
 Creates “user champions” who have
been engaged in the project creation
process and help lead adoption

To be successful with Early Unbiased Feedback
Make it • Easy
• Inexpensive
• Fast
• Part of the process

Questions?
Summary: Early Unbiased Feedback will help you
balance the effort vs. value equation for your users,
leading to quicker adoption

References
 Practical Guide to Controlled Experiments on the Web: Listen to Your
Customers not to the HiPPO, KDD 2007 (http://www.exp-
platform.com/Pages/hippo.aspx)
 Google Design Sprint: http://www.gv.com/sprint/
 https://journal.mozestudio.com/why-we-kick-off-any-new-project-with-a-
design-sprint-5b7177f432b5#.nudz75s7i
 http://blog.elation.com/coaching-blog/cognitive-biases-in-leadership
 http://businessinsider.tumblr.com/post/127644847018/20-cognitive-biases-
that-screw-up-your-decisions
 https://www.fastcompany.com/3048751/the-future-of-work/happy-
employees-are-12-more-productive-at-work
 https://jamesbirchler.com/2016/10/06/lean-startup-practices-write-
hypotheses-you-can-learn-from/
 http://uxpamagazine.org/sustified/
 http://uxpajournal.org/wp-
content/uploads/pdf/JUS_Brooke_February_2013.pdf

Rapid Prototyping and Usability Testing - HUXPA

More Related Content

What's hot (20)

Viewers also liked (9)

Similar to Rapid Prototyping and Usability Testing - HUXPA (20)

Recently uploaded (20)

Rapid Prototyping and Usability Testing - HUXPA