Aect 2018 workshop

User Experience and Usability Testing
Within the Sociotechnical Process –
Methods, Challenges and Pitfalls
Isa Jahnke, Hao He, Yen-Mei Lee, Minh Pham, Gayathri
Sadanala, & Joi Moore

S C H O O L O F I N F O R M AT I O N S C I E N C E S A N D
L E A R N I N G T E C H N O LO G I E S ’
INFORMATION
EXPERIENCE
LABORATORY
University of Missouri
http://ielab.Missouri.edu

Road map
Morning
08.30 Welcome & Agenda
08.45 Warming-up
09.15 Part 1: The four project phases – Theory vs. Experience
10am coffee break
10.15am Part 2: Considerations of how to bring technology into work:
How organization, humans and technology affect each other
12.00 Lunch break

Road map
Afternoon
1.00 Part 3: Introduction into classic methods
• Task-based analysis (Minh)
• Think Aloud (Minh)
• Individual and focus groups interviews (Yen-Mei)
• SUS (Gayathri)
• Expert Review with Nielsen’s Heuristics (Gayathri)
• Prototyping (Hao)
2.15 Part 4: Group work 3-5 participants per group design a study for:
• Client 1: Push (Minh)
• Client 2: Library (Yen-Mei)
• Client 3: AECT registration website (Gayathri)
• Client 4: Media Innovation Space (Hao)
3.15 Groups present their solutions
4.00 Lessons Learned (Discussion of possible collaborations)
4.30pm Happy Ending

WARMING UP: LIVING STATISTICS
• Are you from North, East, South or West within the US, or outside US?
• How many years of Experience in UX/Usability?
Build a row, 1 year or less, to 10 years or more
• For what domain do you use or want to use usability/UX testing
e.g., Education, Service (Insurance, …) Health Care, IT development
• What expectations do you have?
A) Learn UX methods, B) General curiosity, C) Hands-on experience,
D) Interest in technology use for UX, or E) Others
• Do you have a question?

WHAT IS THE IE LAB?
• Founded in 2003
• The Information Experience Laboratory, IE Lab – is a
usability and user experience lab …
• … with the mission to improve learning technologies,
information and communication systems.
The IE Lab conducts research and evaluates digital
technology for clients in diverse fields, for example,
journalism, healthcare, medicine, computer science and
education, and more.

THE IE LAB OFFERS….
• Professional usability and User eXperience
(UX) evaluation services
• Human-centered studies in organizations, e.g.,
how technology impacts existing workflows
• Studies of effectiveness of pedagogical
approaches used in digital platforms

The downside of new technology is that it’s designed by
‘technologists’. But they don’t think like the ordinary user.
The ‘ordinary person’ is your grandmother, doctor, nurse,
patient or student (and so forth). Most people have the
capacity to understand the technology. However, to motivate
them to really use the product: make the technology easy to
use for them!
Note: Top down strategies are not always helpful
because users may create work arounds
when the technology doesn’t fit their workflows

WHY IS USABILITY IMPORTANT? (1)
BENEFITS
Business view
• Provides early evidence of success (allows you to confirm the validity of design early
the development process)
• Improves risk management (lets you test and choose on the design to proceed with)
• Cheaper (decreases cost of development)
• Faster (shortens development timeline)
• Increases sales (better products design and effectiveness)

Design/Research view
• More accurate designs (a more accurate picture of user needs)
• Proof that your design or prototype works (validation of designs, proof of concepts)
• Accurate picture of workflow (know how users work before coding)
• Reduced task time (improved user performance)
• Higher satisfaction (users will be able to find what they need faster)
• Higher motivation (users will be motivated to use and return to your website)
WHY IS USABILITY IMPORTANT? (2)
BENEFITS

WHAT IS USABILITY?
Usability measures the quality of a user’s experience when interacting
with a product or system—e.g., a web site, a software application, mobile
technology.
In general, usability refers to how well users can use a product to achieve their
goals and how satisfied they are with that process/workflow.
• Ease of use
• Satisfaction
• Efficiency of use
• Error frequency and severity
Image Source: www.katzenbergdesign.net

USABILITY FRAMEWORK
Image source: https://www.researchgate.net/figure/Usability-framework-according-
to-ISO-9241-11-ISO-9241-1998_fig1_233428176

PART 1
THE FOUR PROJECT PHASES
1. Assess Needs
2. Design Study
3. Conduct Study
4. Deliver Results

PROCESS
1) Assess Project
• Assessment of the client’s
needs
• Contract
• Graduate researchers
responsibility for the client
study
2) Design Study
• Determination of testing methods
• Organization of experts, resources
and test participants as required to
conduct the study
3) Conduct Study
• Usability study with expert
reviews, user observations, and
interviews
• Data collection and Analysis
4) Deliver Results
• Present results
• Recommendations on:
– The improvement of
usability

Theory vs. Experience
What can go wrong?
Mobile Microlearning Library Usability RJI Prototyping
Assess
needs
• In the beginning, the client
wanted to build up a brand-
new mobile app.
• However, after we did
research, there were a
bunch of mobile learning
‘apps’ out there suitable for
the client’s needs.
• We provided evidences and
references to the client that
allowed the client to make
decision whether or not it
was necessary to develop a
new mobile app.
• We had no problem in
defining needs. The
client came to us with
well-defined needs.
• The client wanted us to
test a library website
with end users through
task-based evaluation.
• At first we thought it
was a usability testing
project and we were
waiting for the client to
give us the product.
• Finally we realized it
was a prototyping
project and we needed
to start everything from
scratch.

Design the
study
Mobile
Microlearning
• When examining
the current
platform: we did
not expect that we
need to build up
several criteria for
the study.
• Thus, we spent a
lot of time on
working on
creating the criteria
based on the
client’s needs.
Library
• We designed
evaluation tasks
based on a shared
prototype from the
client.
• However, the client
was still improving
the same prototype
at the same time.
• This nullified some of
our work because
some issues on that
version of the
prototype no longer
existed.
• We requested an
independent
prototype.
RJI Protoytyping
• At first we created a
design document.
However, it didn’t
work. We switched to
a brain storming
method and draw
ideas in a mind map,
which later became
the site map.
• The site map was
created in MS-Word,
but that caused
problems and display
problems. Then we
switched to XMind, a
mind tool. It worked
well.

Conduct
study
• No training in
how to
analyze
literature
review data
and interview
transcripts.
• Thus, during
the data
analysis
process, the
researchers
spent a lot of
time on
writing/
analyzing.
At first we did not
have prior consent
on how to rate
completion
performance. There
existed discrepancy
among raters in
terms of completion
rate of participants.
• A mind map was not
enough. It showed too
many ideas. We tried to
simplify by learning how
potential users think what
they need, and then we
determined what
functions/ ideas we
should keep and what
should be discarded.
• Data collection was not
conducted through
collecting first-hand data.
Instead they were
collected by IELab from
interview transcriptions
provided by the client.

Deliver
results
• The final report
were completed by
the entire team.
• However, because
each section of the
report was written
by different
researchers, in the
end, the person
who was
responsible for the
combination needs
to clarify the writing
contents with each
team member. This
way was time
consuming.
• We shared results
with the client in the
form of a formal
report. No problems
here.
• Usually, we deliver
evaluation reports
through emails.
• However, due to
safety reason, the
email system blocked
our email with the
prototype to the
client. Therefore, we
had to use a cloud
storage (Google
Drive) to deliver our
deliverables.

PART 2
CONSIDERATIONS OF HOW TO BRING
TECHNOLOGY INTO WORK:
HOW THE ORGANIZATION, HUMANS AND
TECHNOLOGIES AFFECT EACH OTHER

The context matters!
• What is the Sociotechnical Integration Design?
• Sociotechnical Walkthrough (STWT) as a method for UX
studies:
STWT: proofed concept!
“Improving the Coordination of Collaborative Learning with Process
Models” Carell, Herrmann, Menold, in: CSCL2005
THE SOCIO-TECHNICAL …

Problems? Have you ever faced
challenges like this...
• Your organization makes decision for new technology
and you have to work with it
• Technology does not do what you want to do
• Technology does not match task
• Technology does not work when you need it
• You have better tools in mind but your organization don‘t want to hear
• You built “work arounds“ to avoid the system
• …
Technology Tasks

We can engineer/design and control
technical systems
(and technical system rules)…
…but social practices evolves!
We cannot control how people use technology
“Social Systems”, Niklas Luhmann, 1998

SOME THOUGHTS….
“Technology should not be the
driver for teaching-learning
but pedagogy should come
first and then you choose the
right app”
…what do you
think is the
problem with
that?

THE WICKED PROBLEM
• To know the ’right’ technology, you
have to test it and to use it
• To use technology, you have to
change your current social practice
--> a new social
practice
Co-evolutionary growth
of both the Technical and
the Social together

TECHNICAL OR SOCIAL OR
SOCIOTECHNICAL ?
HCI
Person-Tool-
Relationship…
…woohoo really?

SOCIAL OR TECHNICAL OR
SOCIOTECHNICAL ?
Organization
Culture
Situation/
Context

Elements of processes / coordination
– Actors
– Activities
– Interdependencies (between resources)
COORDINATION THEORY
MALONE AND CROWSTON (1990)

• Relating goals/activities, actors and technologies to each other
AND
• Managing interdependencies (e.g., shared objects, make visible
relationships)
COORDINATION THEORY
MALONE AND CROWSTON (1990)
• The result of one activity
can be the input for
another activity

OUR APPROACH:
SHIFT FROM TECHNICAL TO
SOCIOTECHNICAL INTEGRATION DESIGN
The positive impact of a technical system on the Social
depends on its way of being integrated into
organizational processes, competence structures, and
so forth …

ASPECTS OF SOCIO-TECHNICAL
Features of the
technical component
Communication and
cooperation
Work processes,
work flow
Training and
competence
Roles and actors
Ressources
Conditions, events,
exceptions
Interplay between
technical components
Needed: a type of documentation which take all these aspects systematically
into account and provides enough flexibility (for multiple perspectives)
as well as consistency (for the sustainability of decisions)

DOCUMENTATION FOR
SOCIO-TECHNICAL
GRAPHICAL DIAGRAMS AS A DOCUMENTATION
FORM
• Integration of formal and informal structures
• Integration of technical and social aspects
• Integration of ethnographic material
• Tools for editing and presentation
Experience: The success of the documentation depends
essentially on the communication processes
.
Research
since
1997
.
Many
cases
.
Different
fields
.
.
Prof. Thomas Herrmann &
IMTM-IAW team!
www.imtm-iaw.rub.de

STWT
FOUNDER: PROF. DR. THOMAS HERRMANN & TEAM, UNIV. OF BOCHUM,
GERMANY
• Sociotechnical Walkthrough is
• a method to design and develop technology
use in groups (small-big)
• by identifying and documenting current social
practices and then to identify and document
how technologies can help toward new social
practices (processes)
• in a participatory design setting.

COMMUNICATION-ORIENTED INTERVENTION METHOD
(PREPARATION FOR CHANGE MANAGEMENT)
• Structured
• Focused
• Facilitated
• Continuous
• Sustainable
 Socio-technical Walkthrough (STWT)
… communication and
documentation
(aims to integrate technical,
social, organizational,
cultural aspects

OUTCOME OF A STWT
• Decisions about the features of the socio-technical
system – and about the work processes of the future
• Decisions about decisions which are postponed or left
open
• Start of a learning process
• Sustainable description about the characteristics of the
socio-technical system

BACKGROUND OF STWT: COMBINATION
OF…
• Observations…
– Allow for an unobstructed view on a process and allow to easily identify deviations
– Only works for processes that are visible and observed subjects are prone to
behave differently than they would without observation
• Interviews…
– Allow for identifying aspects of processes that cannot be observed
– Only one perspective on a process is considered at a time and ones perspective on
a process might be obstructed or biased.
• Workshops…
– Allow for exchanging perspectives and building common ground
– Scheduling could be hard and only a limited number of people can reasonably
participate

SOCIO-TECHNICAL WALKTHROUGH
THE IDEA BEHIND THE APPROACH
knowledge
gap Stakeholder
(process)
Consultant
(process modeling)
Participatory
collaborative
modeling

Task of the facilitatorPreparing
workshops
Work-
shop 1
Work-
shop 2
Work-
shop n
Developing
or discussing
the work
process and
the models
step-by-step
Asking
prepared
questions
Collecting
problems
comments,
proposals,
documents
Refocus
on the
model
modifying
models,
visualizing
notes
Outcome: models, notes,
Work documents
 Ideas for first changes of work practice
SOCIOTECHNICAL WALKTHROUGH
CORE FACILITATOR ACTIVITIES

EXAMPLE – GRAPHICAL MODELLING
Roles / Actors
Information and instruments
Tasks and Processes

TOOLS
Pieces of Information needed for coordination.
Tasks and Processes

THE EXAMPLE OF ELEVATED - A TOOL FOR STRATEGIC
IMPROVEMENT PLANNING (SIP) IN SCHOOLS
• Schools apply strategic improvement planning / SIP
• New technology called ElevatEd shall help
• We conducted a traditional usability study
• …and sociotechnical integration study (technology development and
preparation for implementation) with 3 teachers and 3 principals
We wanted to know:
1. What kind of activities exist in current SIPs, in what ways are
users (principals, teachers, and others) involved, and what kind of
digital systems are connected to the SIPs workflows?
2. What challenges do principals and teachers face while carrying
out the SIP in schools?
Any ideas
how the
models
look?

STWT PREPARATION
• Get an idea about the context of a process beforehand (e.g.
through interviews or document analysis)
– Start and end of the process
– Circumstances under which process takes place
• Break down overall goal to smaller goals for each workshop
• Set number of workshops and time for each workshop
• Assemble a suitable group
– All stakeholders should be present (including decision makers)
– Potentially invite domain experts or future customers
– It may become necessary to alter group during a modeling project
• Prepare guiding questions for workshop

WHAT MAKES A “GOOD” QUESTION?
• A good guiding question:
– Is open: How, what, why, …
– Makes participants think about their concrete work environment
– Fosters creativity
– Relates a new solution to current practice
– Connects both old and new work process
• Examples:
– Think about a concrete case that you are currently working on. Does this case fit to
the process that is being modeled?
– How could the new software improve your current work process?
– What would help you to work more effectively or efficiently?
– What would help you to improve collaboration with your colleagues?

TIPS -- DURING STWTS SESSIONS
GENERAL FACILITATOR TASKS
• Open and guide discourse
(Overview / background / goals)
• Try to integrate all participants into the discussion:
– All participants are experts, at least for their own practice / perspective
• Encourage critical questioning
• Keep track of the workshop goals
• Stay neutral, do not take a certain position / perspective.
• Ask for specific example

CONCRETE GUIDELINES
• Go through the process step-by-step
• Start with activities, then roles, then entities
• Each contribution should leave a trace in the model
• Use generally accepted terms
• Walk through process parts multiple times (if necessary)
• Show conflicts or diverging perspectives in the model
• Make use of comments and additional flipcharts
• Re-focus participants on the model

GENERAL QUESTIONS
• What happens next?
• What happens before?
• Under which conditions do you do this?
• When do you do something else?
• Who does that?
• Who is involved?
• Which resources do you need to do that?
• What is the result of this?
Questions are generally related to the characteristics of
processes.

RELATION OF STWT TO OTHER METHODS
JAD (Joint Application Design) (Crawford, 1994) Shares the relevance
of workshop/session series such as STWT but not the necessity of diagrammatic
representations of work processes
Scenario based techniques (Carroll, 1995) STWT: focus on diagrams
which cover a variety of possible scenarios, scenarios are an optional means for
illustration whereas STWT does not illustrate optimal but real current practices at
first
Contextual Design (Holtzblatt, 2004) Validates models – also in workshops –
which are provided by an ethnographer, wheras STWT integrates the users as co-designers,
model development and validation together, and focusses on work processes
MUST (Method for Participatory Design) (Kensing et al., 1996)
intervention is seen as a communication process similar to STWT, different kind of modelling
Analysis of task procedures (Jonassen, 1986) task analysis process:
inventorying, describing, selecting, sequencing, and analyzing tasks, differentiated analysis
procedures
Herrmann, Kunau, Loser, Menold, in: PD 2004

IF THERE IS TIME
LET‘S DO SEEME AND MODELING TOGETHER

Role – carried out by single persons or
groups like departments. They usually
come with a set of rights and
responsibilities.
Activity – tasks which are carried out by
roles. They usually use entities or modify
them.
Entity – static aspects of processes (e.g.
objects, data, systems)
activity [1]
role
document
technical
system
activity [2]
SEEME BASICS
BASIC ELEMENTS

BEFORE WE START MODELING….
WHAT ABOUT THE MODELING NOTATION?
• Explain basic constructs of the SeeMe notation…
– At the beginning of the
workshop or
– When an element appears
for the first time

expects
something from
role [1]
activity [1]
role [2]
activity [2]
entity [1] entity [2]
These standard definitions can be altered by attaching
alternate labels to the respective relations.
carries
out
belongs
to
leads to
affects
can be
described by
modifies
is used
by
relates to
SEEME BASICS
RELATIONS

ELEMENTS OF MODELING
NOTATIONS
• Organizational structure
• Roles (e.g. engineer, project manager)
• Rights and responsibilities
• Functions
• Workflow
• Executed by roles or systems
• Same granularity as other functions
• Information
• Data and containers (artifacts)
• Resources
• In- and Out-put for (Sub-)processes
• Control
• Branches and conditions

…let’s model somethingWebeditor
https://cme.iaw.rub.de/
We want to design and develop a tool for AECT 2018 that includes the
online program, conference registration and communication.
What activities should participants be able to conduct with such a
tool?

TIPS - STRUCTURE YOUR MODEL
Do not focus on structure too much!

Document management
structuringeliciting
updating
Quality assurance
TIPS -- DON´T TRY TO BE OVERLY
COMPLETE

comments sketching
TIPS -- USE INFORMAL ELEMENTS AND
VAGUENESS
…while still using model elements!

USING THE SEEME EDITOR
PRESENTING MODELS
• Prepare a model for presentation by hiding
elements using the hide tool
• Showing elements
using the grey button at
the bottom of an
element

SPECIFICS OF VISUAL MODELING
NOTATIONS
• Consist of elements (visual shapes) and relations
(connections between them)
• A syntax describes how these elements may be
interconnected with each other
• A semantic describes their meaning in relation to real life
phenomena
Some modeling notations contain multiple diagram types (e.g. UML)
altersprogrammer code

SIX CLASSIC METHODS
• Task-based analysis (Minh)
• Think Aloud (Minh)
• Focus groups interviews (Yen-Mei)
• System Usability Score, SUS (Gayathri)
• Expert Review with Nielsen’s Heuristics (Gayathri)
• Prototyping (Hao)

The process of learning about ordinary users by observing them in action to
understand in detail how they perform their tasks and achieve their intended
goals (Usability.org).
It measures:
• based on what your goals are with the system: if system helps your users to
achieve and the goals efficiently (by conducting/performing special defined
tasks)
• what users actually do to achieve those goals
• the user’s experience with the system
• how users are influenced by their physical environment
• how users’ previous knowledge and experience influence the use of the
system:
– what they think about the workflow
Task-based Analysis (Minh Pham)
(Hackos, & Redish, 1998)

• 10-12 tasks per scenario
• Several scenarios at a stage of design
• Ask users about their own scenarios
• Notes about how users would complete tasks
• Comparisons between notes and observations
Scenarios
You are traveling to
Seattle for your job next
week and you want to
check on the amount you
can be reimbursed for
meals and other
expenses.
(Usability.org)

“A use case is a written description of how users will perform tasks on a
software. It outlines, from a user’s point of view, a system’s behavior as it
responds to a request. Each use case is represented as a sequence of simple
steps, beginning with a user's goal and ending when that goal is fulfilled.”
• Who is using the software
• What the user wants to do with the system
• The user's goal with the system
• The steps the user takes to accomplish a particular task
• How the software should respond to an action
Use cases
(Usability.org)

“In a thinking aloud test, you ask test participants to use the system while
continuously thinking out loud — that is, simply verbalizing their thoughts as
they move through the user interface. “ (Nielsen, 2012)
To run a basic thinking aloud usability study, you need to do only 3 things:
1. Recruit representative end users
2. Give them representative tasks to perform
3. Let the users do the talking
Think Aloud (Minh Pham)

A qualitative research technique
which involves “conducting intensive
interviews with a small number of
respondents to explore target
audiences’ perspectives on a
particular idea, program or situation
(Boyce & Neale, 2006) .
Reference: Boyce, C. & Neale, P. (2006) “Conducting in-depth Interviews: A Guide for Designing and Conducting In-Depth
Interviews”, Pathfinder International Tool Series
Resource: http://thegolfclub.info/6a6f62/job-interview-icon.html
Interviews (Yen-Mei Lee)

TYPES OF INTERVIEW
Reference: Research Methodology. Retrieved from: https://research-methodology.net/research-
methods/qualitative-research/interviews/
Structured Interview
• A series of pre-determined questions.
• All interviewees answer in the same
order.
Unstructured Interview
• No questions are prepared prior to the interview
• In an informal manner
Semi-structured Interview
• A set of same questions are answered by all interviewees.
• Additional questions might be asked to clarify and/or further expand certain
issues.

Individual Interview
• Talk to only one person at a time (30 minutes to an hour)
• Probe the interviewee’s attitudes, beliefs, desires, and experiences.
• Face-to-face, by phone or video conference, or via instant messaging
system.
• Dive deeper
Focus Group Interview
• 5 to 10 participants (60 to 90 minutes)
• Focus on a specific, focused discussion topic
• Has a trained leader, or facilitator
• Gather a range of ideas and feelings (enrichment of responses)
• When recruiting: Specific traits or characteristics (Age, Occupation,
Experience, Education, Ethnicity, etc. )
Reference: Usability.gov. Retrieved from: https://www.usability.gov/how-to-and-tools/methods/focus-groups.html
Individual Interview vs. Focus Group Interview

• Top 16 Qualitative Data Analysis Software in 2018
(Nvivo, ATLAS.ti, MAXQDA, etc.)
• Top 21 Free Qualitative Data Analysis Software
INTERVIEW ANALYSIS:
TECHNIQUE & SOFTWARE
• Thematic content analysis: identify topics/themes from user’s view
• Narrative analysis

What is SUS?
• System Usability Scale
• Survey tool - Data collection instrument
- user’s subjective rating of a product’s
usability.
Why SUS?
• easy to understand: single score
• flexible to assess
• easy to use
• cost-effective
Benefits
• sample sizes with reliable results
• Is valid
Things to keep in mind
• Complex scoring system
• normalize to percentiles
• not a diagnostic
SYSTEM USABILITY SCALE
(Gayathri Sadanala)

Source: https://www.measuringux.com/SUS.pdf
Scoring: 0-40
Interpreting the scores
Converting to a new number
Odd rows: +1
5- Even rows
Sum*2.5 ==> 0 to 100
Percentile ranking:
Below average
---SUS score 68 (50th percentile average)---
Above average
10 item questionnaire with 5 response
options

• Potential usability issues and strengths
• UX Expert - Exposure to the real user behavior
(EXPERTISE)
Past experience + Knowledge
Usability heuristics + cognitive psychology + human information
behavior
Deliver design recommendations
EXPERT REVIEW
WITH NIELSEN’S HEURISTICS
(Gayathri Sadanala)

NIELSEN HEURISTICS
https://www.designprinciplesftw.com/collections/10-usability-heuristics-for-user-interface-design

Components
of an Expert
Review
• List of usability strengths
• List of usability problems
• Severity ratings
• Recommendations
When
to do
• Any time in the
design cycle
• Iteratively
• At the phase end

PROTOTYPING
A prototype is an
early release of a
product built to
test whether a
concept, a process
or something
works in the
desired way.
(Hao He)

PROTOTYPE VS MODEL
Model vs Prototype
anything (animal, human,
house, car, virtual,
conceptual, etc.)
Source a product
smaller than real Size same or similar size
No
Carry the
function?
Yes
to show the appearance Purpose
to identify any flaws for
correction

TYPES OF PROTOTYPE
High
Medium
Low

PROCEDURE OF PROTOTYPING
WireframeSketch Mockup Prototype

TOOLS FOR PROTOTYPING
Prototype
Mockup
Wireframe
Sketch
pen
Adobe XD
Balsamiq
Axure RP
paper
OneNote
Mockplus
Moqups
PowerPoint
Wireframe.cc

2.15PM GROUP WORK
Client 1: Push Project (Minh)
Client 2: Library Project (Yen-Mei)
Client 3: AECT website/registration (Gayathri)
Cilent 4: Prototype of Media Innovation Space Project (Hao)

Client 1
What the client wants…
I am an IT Manager of a public city library and would like you to
help us evaluate our new library website which is
underdevelopment.
In the early stage, can you provide a execution proposal to help
us understand what kind of process and methods you will apply
for the newly developed library website?
Information: Now I have six types of target users (personas)
retrieved from the library statistic system, including Digitarians,
Stay Connected, Page Turner, Bedtime Stories, Bright Future,
Library Staff.
(Yen-Mei)

Background: Push is an open source mobile app for news outlets to create
their own mobile news apps (two apps named Meydan TV and Bivol )
I want to know whether the apps are usable.
Your task:
Please have a look at the apps and propose a usability study design,
specifying methods, data collection, data analysis, and results/ deliverables.
Client 2:
What the client wants
(Minh)

Background: Client is the AECT registration website
I want to know whether the website is user-friendly , easy to use, and functionable.
Your task:
Propose a study design, specifying methods, data collection, data analysis, and how to
present the study results.
Client 3:
(Gayathri)

Client 4:
(Hao)
Your task: prototype!

BACKGROUND:
Copyright 2017 Dr. Michelle Ferrier All Rights
SUPPORT TO START A
COMPANY
MENTOR STUDENT
ENTREPRENEURS
TO EXPAND STUDENTS’
NETWORK
An online platform for journalist students to ...
Client 4: Media Innovation Space (MIS)
(Hao)

MIS – USER NEEDS
• collaboration
• engagement
• feedback
• hands-on activities
• LMS
• marketing
• mentoring
• network
• practice
• profile
• resources
• showcase
• story telling
• supportive
atmosphere
• training

MIS – FEATURES
NEWS, TRENDS, AND
RESOURCES
MEMBERSHIP SERVICES CAREER INFO AND
TRAINING
COMMUNICATION
AND CONNECTION
ACTIVITIES CONTENT
MANAGEMENT SYSTEM

3.15PM
GROUPS PRESENTING FIRST

WHAT WE ACTUALLY DID…
• Task analysis (Create 8-10 tasks for each target audience)
• Think-aloud
• User Satisfaction Survey (SUS)
• Technology: Morea software
Client 1: Library project (Yen-Mei)

Methods
• Expert Design Review
• Task based usability evaluation.
• Mobile device test
Deliverables
• Design Review report
• Actionable recommendations
• A document with the listing of each of the five mobile device
specifications and errors, issues or concerns pertaining to each of the
devices tested
What we did….
Client 2: Push (Minh)

What IE lab would propose to do
Study Design:
– Task based Interface Evaluation
Methods:
– Think aloud
– Interview protocol
– SUS
Data Collection and Analysis:
– Quantitative and qualitative
Deliverables:
– Review report of usability problems and
– Design recommendations
Client 3: AECT website (Gayathri)

MIS – IE LAB EXAMPLE
Home
Profile My Project Community Accelerator Support
Sign up/Log in
Client 4: What IE Lab did (Hao)

MIS Prototype

4.00PM
LESSONS LEARNED
FURTHER COLLABORATION?

Happy Ending!
 Thank you
jahnkei@Missouri.edu
Dr. Isa Jahnke, Associate Professor and Director of the IELab

Aect 2018 workshop

More Related Content

What's hot (20)

Similar to Aect 2018 workshop (20)

Recently uploaded (20)

Aect 2018 workshop