SlideShare a Scribd company logo
Copyright © Sebbe Isaac Kurian
Heuristic Evaluation
of edX Website’s Discussion Forum
(based on Nielsen's 10 Usability Heuristics)
Heuristic Evaluation to identify usability issues in the edX website’s
(www.edx.org) Discussion Forum – based on Jakob Nielsen's 10 Usability
Heuristics for User Interface Design.
Sebbe Isaac Kurian, January 2018.
Copyright © Sebbe Isaac Kurian - Page 2
Contents
Executive Summary 3
Introduction 4
Findings and Recommendations 6
Conclusion 26
Reference: Nielsen’s Usability Heuristics 28
Appendix: List of Usability Issues 30
Copyright © Sebbe Isaac Kurian - Page 3
Executive Summary
edX (www.edx.org) is a MOOC (Massive Open Online Course) provider, serving
learners worldwide. Each course on edX has a Discussion Forum. This Heuristic
Evaluation was done to determine the usability of edX's Discussion Forum.
The Discussion Forum was evaluated against Nielsen's “10 Usability Heuristics for
User Interface Design”. 15 usability issues were identified, and each of their severity
assessed on a 1 to 4 scale.
The 10 most severe of these usability issues were examined in detail.
They are:
1. Absence of an in-place text editor for creating and editing Posts, Responses,
and Comments.
2. Error handling of the mandatory input fields when adding a new Post is not
optimal.
3. While entering text, cannot start a new line or paragraph by pressing the
“Enter” key.
4. Cannot cancel the process of adding a Response or Comment, once started.
5. Date/Time of editing is not shown in the Posts, Responses, or Comments.
6. When inserting an image in the Posts, Responses, or Comments, the error
handling is not optimal.
7. The process of setting a topic when adding a Post, is not optimal.
8. There is no optimum character limit set for the Title of a Post.
9. Error handling when submitting a new Response or Comment, is not optimal.
10. The More button in a published Comment may overlap with the Comment’s
content.
For each of these issues, suggestions to improve the usability are recommended.
Lastly, the results are summarized, and the limitations of this evaluation clarified.
Copyright © Sebbe Isaac Kurian - Page 4
Introduction
About edX
edX (www.edx.org) is a MOOC (Massive Open Online Course) provider, offering
high-quality courses from top universities and institutions, to learners all over the
world. Founded by Harvard University and MIT in 2012, edX is non-profit and open-
source. Open edX is the open-source platform that powers edX courses.
Each course on edX has a Discussion Forum associated with it. Being fully online,
this Discussion Forum is important for an interactive student experience. On the
Discussion Forum, the learners, course staff, and community teaching assistants
interact with each other. They can browse and contribute (ask/answer/share) to
course discussions.
Why Evaluate?
Efficient and effective learning are key goals of edX. As the core element facilitating
interaction in a course, the usability of the Discussion Forum is crucial for efficient
and effective learning.
Heuristic Evaluation is a cost effective and easy-to-use UX research method, that
can identify usability issues in a user interface. It is a usability inspection technique
which involves evaluating the user interface against a set of recognized usability
principles (also known as heuristics).
Copyright © Sebbe Isaac Kurian - Page 5
Evaluation Procedure
For this evaluation, the “10 Usability Heuristics for User Interface Design”, formulated
by Jakob Nielsen (Nielsen 1994a) was used. They are:
1. Visibility of System Status.
2. Match between System and the Real World.
3. User Control and Freedom.
4. Consistency and Standards.
5. Error Prevention.
6. Recognition rather than Recall.
7. Flexibility and Efficiency of Use.
8. Aesthetic and Minimalist Design.
9. Help Users Recognize, Diagnose, and Recover from Errors.
10. Help and Documentation.
An overview of these Heuristics is given in the Reference section of this document.
15 usability issues were identified. The evaluation determined usability issues in
terms of, which heuristic(s) from the above list is(are) being violated, and the severity
of violation. The severity was assessed according to the following scale:
1 = Cosmetic Problem. No real usability impact. (It is just annoying).
2 = Minor Usability Problem. Fix it, if there is time.
3 = Major Usability Problem. Important to fix.
4 = Usability Catastrophe. Must fix. (Prevents Users from completing the task).
These 15 issues were compiled into a prioritized list, ranked in decreasing order
of severity (Refer Appendix).
The top 10 usability issues in this list were examined in detail. These, along with
the suggestions for improvement for each, are given in the following section,
Findings and Recommendations.
Copyright © Sebbe Isaac Kurian - Page 6
Findings and Recommendations
Finding-1
Absence of an in-place text editor for creating and editing Posts, Responses, and
Comments.
Severity: 3.
Heuristics Violated: 4. Consistency and Standards.
Description:
This relates to the “Add a post/response/comment”, and “Editing
post/response/comment” interactions. For adding and editing “Posts”, “Responses”,
or “Comments” the Users enter/edit/format text in a Text Entry Box.
The resulting formatted text that will be published, is displayed in a separate
“Preview” box under it.
Figure 1.1: Action and Result happening in Different Places.
Copyright © Sebbe Isaac Kurian - Page 7
This violates Heuristic 4 - Consistency and Standards (Nielsen 1994a), according
to which, conforming to universally followed conventions increase the usability of an
interface – by taking advantage of the Users’ previous experience and existing
knowledge. The common practice in most text editors is, in-place editing and
formatting of text – where the action and result happens in the same place. Making
the User switch attention between the separate areas – do the action in one place,
then view the result in another place – increase the perceived complexity of the
interface. Many Users would find this back-and-forth switching confusing.
Suggestions for Improvement:
Provide an in-place text editing interface, like in Windows WordPad, Mac TextEdit,
MS-Word, etc.
Figure 1.2: In-Place text editing in Windows WordPad.
Copyright © Sebbe Isaac Kurian - Page 8
Finding-2
Error handling of the mandatory input fields when adding a new Post is not optimal.
Severity: 3.
Heuristics Violated: 1. Visibility of System Status.
5. Error Prevention.
9. Help Users Recognize, Diagnose, and Recover from Errors.
Description:
This pertains to the “Add a Post” screen. The field labelled “Title”, and the text box
labelled “Your question or idea” are required to be filled by the User. The
“(Required)” note appended to the labels, is not highlighted enough visually to get
Users’ attention, and so could easily be overlooked. This violates Heuristic 1 -
Visibility of System Status (Nielsen 1994a), according to which, the interface
should support discoverability of the actions that Users are supposed to take, by
informing them appropriately.
Figure 2.1: Error Handling of required fields in the "Add a Post" screen.
Copyright © Sebbe Isaac Kurian - Page 9
Users can skip either one or both fields without getting any feedback, until they click
the “Submit” button. This again violates Heuristic 1 - Visibility of System Status,
which advocates appropriate and timely feedback to Users. This also violates
Heuristic 5 - Error Prevention (Nielsen 1994a), because the interface is not
designed in such a way, so as to “prevent the problem from occurring in the first
place” – here the required fields can be skipped without immediate warning.
When error does occur, the Error Notification is not sufficiently highlighted, and does
not effectively direct the Users’ attention to the place(s) where the error occurred.
Also, if the User skips the text box, the Error Message says – “Body can’t be empty”
– a language usage that some Users may not understand. This violates Heuristic 9 -
Help Users Recognize, Diagnose, and Recover from Errors (Nielsen 1994a),
according to which, “error messages should be expressed in plain language”, and
should help the Users by clearly indicating why, where, and how to solve it.
Suggestions for Improvement:
The required fields should be indicated more strikingly. There should be in-process
checking and feedback, to prevent errors before the “Submit” button is clicked. The
Error Notification should help the Users to solve the problem easily. An example of
this is the “Create an account” interaction in the www.facebook.com landing page.
Figure 2.2: The "Create an account" interaction of Facebook.
Copyright © Sebbe Isaac Kurian - Page 10
Finding-3
While entering text, cannot start a new line or paragraph by pressing the “Enter” key.
Severity: 3.
Heuristics Violated: 4. Consistency and Standards.
10. Help and Documentation.
Description:
In the text entry box used for adding and editing Posts, Responses, and Comments,
it is not possible for Users to start a new line or new paragraph, by pressing the
“Enter” key. Pressing the “Enter” key gives a blank space.
Figure 3.1: Pressing "Enter" key in the text box.
This behavior violates Heuristic 4 - Consistency and Standards (Nielsen 1994a),
because the common standard practice of pressing the “Enter” key for a new
line/paragraph is not followed.
Copyright © Sebbe Isaac Kurian - Page 11
On the other hand, pressing the “Enter” key twice in a row, starts a new paragraph.
And for starting a new line, the User must press the “Space” bar twice followed by
the “Enter” key. These are not standard behavior in text editors – again a violation of
Heuristic 4 - Consistency and Standards. Also, these key combinations are not
communicated to the Users – a violation of Heuristic 10 - Help and Documentation
(Nielsen 1994a), which advocates providing help focused on the User’s task.
Suggestions for Improvement:
It would be ideal to support the standard text-entry practice, of pressing the “Enter”
key once for a new line/paragraph. If for any specific reason, uncommon key
combinations are required for this, that information should be communicated to the
User via a help option. For example, like the help provided in the Notepad++ editor.
Figure 3.2: Help option in the Notepad++ editor.
Copyright © Sebbe Isaac Kurian - Page 12
Finding-4
Cannot cancel the process of adding a Response or Comment, once started.
Severity: 3.
Heuristics Violated: 3. User Control and Freedom.
4. Consistency and Standards.
Description:
When Users are viewing a “Post”, the can give a “Response” to it, as well as
“Comment” on any “Response” already given. If the Users click to add a Response
or Comment, and then decide not to save and publish it – they cannot cancel it.
There is only a “Submit” button for saving and publishing Responses and Comments
– No “Cancel” button is provided. This is a violation of Heuristic 3 - User Control
and Freedom (Nielsen 1994a), which advocates the need to provide a clearly
marked “Emergency Exit” to leave an unwanted state, and support Undo.
Figure 4.1: There is no "Cancel" button when adding a Response or Comment.
On the other hand, while adding a Post (in the “Add a Post” screen), there is a
“Cancel” button along with the “Submit” button. So, not providing a “Cancel” button
for Responses and Comments, also violates Heuristic 4 - Consistency and
Standards (Nielsen 1994a), because the interface is not consistent within the
system, and may confuse some Users.
Copyright © Sebbe Isaac Kurian - Page 13
Figure 4.2: There is a "Cancel" button when adding a Post.
Suggestions for Improvement:
Provide a “Cancel” button when adding Responses and Comments – like the one
given when adding a Post. This would give Users the freedom to exit the process if
so desired.
Copyright © Sebbe Isaac Kurian - Page 14
Finding-5
Date/Time of editing is not shown in the Posts, Responses, or Comments.
Severity: 3.
Heuristics Violated: 1. Visibility of System Status.
Description:
When Users submit a new Post, Response to a Post, or Comment to a Response,
the duration of time elapsed since submission is displayed with the
post/response/comment – in minutes, hours, days, months, etc.
Figure 5.1: Time elapsed since submission of a Post, Response, and Comment.
Copyright © Sebbe Isaac Kurian - Page 15
If Users edit their post/response/comment later on, this displayed time period is not
updated to show the time when editing was done. So, there is no way of knowing if
any Post, Response, or Comment is the original version, or an edited version.
This could compromise the integrity of the information displayed, when a Post having
Response(s), or a Response with Comment(s) is edited. Because, then the
Response(s) and Comment(s) will pertain to the respective original Post or
Response, and not to the edited content – and the Users who are viewing/reading it
will not know about this.
This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), according
to which, the Users should be informed about the current system state – which in this
case, is the time of editing of the post/response/comment.
Suggestions for Improvement:
Ideally, both the date and time of submission (not the duration of time elapsed since
submission), and the date and time of editing (if done), should be given in the Posts,
Responses, and Comments.
Copyright © Sebbe Isaac Kurian - Page 16
Finding-6
When inserting an image in the Posts, Responses, or Comments, the error handling
is not optimal.
Severity: 3.
Heuristics Violated: 1. Visibility of System Status.
2. Match between System and the Real World.
5. Error Prevention.
9. Help Users Recognize, Diagnose, and Recover from Errors.
Description:
When adding or editing a Post, Response, or Comment, Users can insert images in
it. Clicking the button for inserting image in the tool bar of the text box (or pressing
Ctrl-G), pops-up an “Insert Image” dialog box. Users can upload an image from their
computer, or give the URL of an image.
Figure 6.1: Dialog box for inserting Image.
Copyright © Sebbe Isaac Kurian - Page 17
The two different use cases were evaluated – uploading an image, and giving the
URL of an image.
1. Trying to upload a big sized image file – gives a terse pop-up error message.
Figure 6.2: Error message pop-up on trying to upload a big image file.
This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), because
the interface does not inform the Users beforehand, about the maximum file size
limit. This violates Heuristic 5 - Error Prevention (Nielsen 1994a), because the
system does not try to prevent this problem from occurring, before initiating the
upload. This violates Heuristic 2 - Match between System and the Real World
(Nielsen 1994a), because the error message uses system-oriented terms, instead of
communicating the problem in User understandable language. This violates
Heuristic 9 - Help Users Recognize, Diagnose, and Recover from Errors
(Nielsen 1994a), because the error message is not in plain language, and does not
suggest a constructive solution for the Users.
2. Entering an invalid URL of an image file that does not exist – gives no error.
This violates Heuristic 1 - Visibility of System Status, because the interface does
not inform the User about what is happening, through appropriate feedback. This
violates Heuristic 9 - Help Users Recognize, Diagnose, and Recover from
Errors, because the interface does not give any error – so, the Users cannot
recognize, or diagnose, or recover from this error.
Copyright © Sebbe Isaac Kurian - Page 18
Suggestions for Improvement:
Inform the Users beforehand, about the file size limit for uploads. Check the file size
on file selection, and notify the User in case of exceeding the size limit, before
initiating the upload process.
If the User enters an image URL, check its availability, and inform the User about
any problems.
In both the above cases, give enough information in plain language, so that Users
can understand what is happening, and know what to do. Figure 6.3, shows an
example of a good error message “expressed in plain language”, that “precisely
indicate the problem”, and “constructively suggest a solution”.
Figure 6.3: An example of a good error message.
Copyright © Sebbe Isaac Kurian - Page 19
Finding-7
The process of setting a topic when adding a Post, is not optimal.
Severity: 2.
Heuristics Violated: 1. Visibility of System Status.
5. Error Prevention.
Description:
In the “Add a Post” screen, the “Topic area” field of the Post is set as “General” by
default. As the input field is shown as already filled-up, the User may not pay
attention to it and overlook it – causing the Post to be submitted under the “General”
topic area by default. This may not be what the User intended.
Figure 7.1: Topic area of a Post set as "General" by default.
Copyright © Sebbe Isaac Kurian - Page 20
This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), because it
conveys potentially misleading information about the system to the User – seeing an
input field as already filled, the User who is scanning the interface may assume that
there is nothing more to do, and skip it. This also violates Heuristic 5 - Error
Prevention (Nielsen 1994a), because instead of preventing a problem from
occurring in the first place, this could cause an error to happen.
Suggestions for Improvement:
Use a dummy placeholder like “Select a Topic”, to indicate the drop-down list of
available Topic areas. This will prevent the Post from being submitted under an
unintended Topic area.
Also, follow the Suggestions for Improvement given for Finding-2, on how to
effectively present the required input fields. This will prevent the Users from skipping
the “Topic area” field.
Copyright © Sebbe Isaac Kurian - Page 21
Finding-8
There is no optimum character limit set for the Title of a Post.
Severity: 2.
Heuristics Violated: 5. Error Prevention.
Description:
When adding or editing a Post, the User can enter a whole page of characters in the
“Title” field of a Post. If the Post is then submitted/updated, this full title is displayed
in the Post’s thumbnail view on the left panel. This causes all previous Post
thumbnails to scroll out of view.
Figure 8.1: Abnormally long Title in the Post's thumbnail view.
This violates Heuristic 5 - Error Prevention (Nielsen 1994a), because the interface
does not prevent this problem from occurring in the first place.
Suggestions for Improvement:
Set an optimum character limit for the Title, inform the Users about this limit, and
prevent them from exceeding it.
Long Title
Copyright © Sebbe Isaac Kurian - Page 22
Finding-9
Error handling when submitting a new Response or Comment, is not optimal.
Severity: 2.
Heuristics Violated: 1. Visibility of System Status.
5. Error Prevention.
9. Help Users Recognize, Diagnose, and Recover from Errors.
Description:
If the Users submit a new Response or Comment, without any content in the text
box, nothing happens – No feedback, No error.
Figure 9.1: Adding a new Response or Comment without content.
Copyright © Sebbe Isaac Kurian - Page 23
This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), because
the interface does not give any feedback to the Users, informing them about what is
happening. This violates Heuristic 5 - Error Prevention (Nielsen 1994a), because
the interface does nothing to prevent this problem from occurring. This violates
Heuristic 9 - Help Users Recognize, Diagnose, and Recover from Errors
(Nielsen 1994a), because the interface does not give an error message – so, the
Users cannot recognize, or diagnose, or recover from this error.
Suggestions for Improvement:
Follow the Suggestions for Improvement given for Finding-2, on how to effectively
present the required input fields. This would clearly indicate to the Users that the text
box should not be blank.
Also, follow the Suggestions for Improvement given for Finding-4, by providing a
“Cancel” button. This would give Users the freedom to exit the process if so desired.
Copyright © Sebbe Isaac Kurian - Page 24
Finding-10
The More button in a published Comment may overlap with the Comment’s content.
Severity: 2.
Heuristics Violated: 1. Visibility of System Status.
4. Consistency and Standards.
Description:
When viewing a published Post with Response(s) and their Comment(s), the
collapsed “More” button in a Comment may overlap with the Comment’s content. If
this happens, the Users may not notice this button at all. The content does not
overlap the button in Posts or Responses – this happens only in Comments.
Figure 10.1: Collapsed More buttons in Post, Response, and Comment.
Copyright © Sebbe Isaac Kurian - Page 25
This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), because
the interface does not clearly inform the Users about available actions, when the
button and content overlap.
This also violates Heuristic 4 - Consistency and Standards (Nielsen 1994a),
because this is an instance of inconsistency within the system. This possibility of
overlapping is not there in Posts or Responses, as illustrated in Figure 10.1. This
could happen only in Comments.
Suggestions for Improvement:
Keep the content of the Comment clear from its collapsed “More” button – like it is
being done in Posts and Responses (as shown in Figure 10.1).
Copyright © Sebbe Isaac Kurian - Page 26
Conclusion
The Heuristic Evaluation uncovered quite a few usability issues in the edX website's
Discussion Forum.
The most severe of them are:
1. Absence of an in-place text editor for creating and editing Posts, Responses,
and Comments.
2. Error handling of the mandatory input fields when adding a new Post is not
optimal.
3. While entering text, cannot start a new line or paragraph by pressing the
“Enter” key.
4. Cannot cancel the process of adding a Response or Comment, once started.
5. Date/Time of editing is not shown in the Posts, Responses, or Comments.
6. When inserting an image in the Posts, Responses, or Comments, the error
handling is not optimal.
7. The process of setting a topic when adding a Post, is not optimal.
8. There is no optimum character limit set for the Title of a Post.
9. Error handling when submitting a new Response or Comment, is not optimal.
10. The More button in a published Comment may overlap with the Comment’s
content.
Suggestions for improvement have been recommended to address these interface
problems.
By implementing these suggestions, the usability of the Discussion Forum would
improve. This should significantly enhance the learning experience of students, and
in turn help edX meet the key goals of efficient and effective learning.
Copyright © Sebbe Isaac Kurian - Page 27
Limitations
This Heuristic Evaluation was done by a single evaluator.
According to Jakob Nielsen, single evaluators find on an average only 35 percent of
the usability problems in interfaces.
(Refer: “How to Conduct a Heuristic Evaluation” by Jakob Nielsen, January 1, 1995.
URL: https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation).
He points out that, since different evaluators tend to find different problems, better
results are achieved by combining the evaluations from several evaluators.
Considering the ratio of the benefits to the costs, he recommends 3 to 5 evaluators
for the best, cost effective results.
Copyright © Sebbe Isaac Kurian - Page 28
Reference
10 Usability Heuristics for User Interface Design
Nielsen, J. (1994a). Enhancing the explanatory power of usability heuristics. Proc.
ACM CHI'94 Conf. (Boston, MA, April 24-28), 152-158.
Retrieved from https://www.nngroup.com/articles/ten-usability-heuristics/
Copyright © 2005 by Jakob Nielsen. ISSN 1548-5552
Jakob Nielsen's 10 general principles for interaction design are called "heuristics"
because they are broad rules of thumb and not specific usability guidelines.
1. Visibility of System Status
The system should always keep users informed about what is going on, through
appropriate feedback within reasonable time.
2. Match between System and the Real World
The system should speak the users' language, with words, phrases, and
concepts familiar to the user, rather than system-oriented terms. Follow real-
world conventions, making information appear in a natural and logical order.
3. User Control and Freedom
Users often choose system functions by mistake and will need a clearly marked
"emergency exit" to leave the unwanted state without having to go through an
extended dialogue. Support undo and redo.
4. Consistency and Standards
Users should not have to wonder whether different words, situations, or actions
mean the same thing. Follow platform conventions.
Copyright © Sebbe Isaac Kurian - Page 29
5. Error Prevention
Even better than good error messages is a careful design which prevents a
problem from occurring in the first place. Either eliminate error-prone conditions
or check for them and present users with a confirmation option before they
commit to the action.
6. Recognition rather than Recall
Minimize the user's memory load by making objects, actions, and options
visible. The user should not have to remember information from one part of the
dialogue to another. Instructions for use of the system should be visible or
easily retrievable whenever appropriate.
7. Flexibility and Efficiency of Use
Accelerators — unseen by the novice user — may often speed up the
interaction for the expert user such that the system can cater to both
inexperienced and experienced users. Allow users to tailor frequent actions.
8. Aesthetic and Minimalist Design
Dialogues should not contain information which is irrelevant or rarely needed.
Every extra unit of information in a dialogue competes with the relevant units of
information and diminishes their relative visibility.
9. Help Users Recognize, Diagnose, and Recover from Errors
Error messages should be expressed in plain language (no codes), precisely
indicate the problem, and constructively suggest a solution.
10. Help and Documentation
Even though it is better if the system can be used without documentation, it may
be necessary to provide help and documentation. Any such information should
be easy to search, focused on the user's task, list concrete steps to be carried
out, and not be too large.
Copyright © Sebbe Isaac Kurian - Page 30
Appendix: List of Usability Issues
Table 1: List of Usability Issues found in the edX website’s Discussion Board.
Issue
#
Description
System
Location
Heuristics
Violated
Severity
1. For adding and editing “Posts”, “Responses”,
or “Comments” the Users enter/edit/format
text in a Text Entry Box. The resulting
formatted text that will be published, is
displayed in a separate “Preview” box under
it. This could be confusing for many Users, as
the common practice in most text editors is
that, the result of editing and formatting is
visible where it is done.
“Add a post/
response/
comment”,
and “Editing
post/
response/
comment”
interactions.
4. Consistency
and Standards.
3
2. Error handling of the input field labelled
“Title”, and the Text Entry Box labelled “Your
question or idea” is not optimal. Both are
“required” fields that cannot be left blank.
The “(Required)” note appended to the labels
is not highlighted enough to get Users’
attention, and so could easily be overlooked.
Users can skip either one or both of these
fields without getting any feedback, until they
click the “Submit” button. Also, the error
messages are not sufficiently highlighted, and
does not effectively direct the Users’ attention
to the place(s) where the error occurred.
“Add a Post”
screen.
1. Visibility of
System Status.
5. Error
Prevention.
9. Help Users
Recognize,
Diagnose, and
Recover from
Errors.
3
Copyright © Sebbe Isaac Kurian - Page 31
Issue
#
Description
System
Location
Heuristics
Violated
Severity
3. In the Text Entry Box, it is not possible for
Users to start a new line, or new paragraph,
using the common practice of pressing the
“Enter” key. Pressing the “Enter” key once,
gives a blank space.
Pressing the “Enter” key twice in a row starts
a new paragraph. This is not common
practice in text entry. Also, this feature is not
communicated to the User.
“Add a post/
response/
comment”,
and “Editing
post/
response/
comment”
interactions.
4. Consistency
and Standards.
10. Help and
Documentation.
3
4. There is a “Submit” button for saving and
publishing “Responses” and “Comments”, but
there is no “Cancel” button. So, if Users click
to add a “Response” or “Comment”, and then
decide not to proceed, they are stuck. They
cannot undo. Meanwhile, for “Posts”, both
“Submit” and “Cancel” buttons are provided.
“Add a
response” and
“Add a
comment”
interactions.
3. User Control
and Freedom.
4. Consistency
and Standards.
3
5. If Users edit their “Posts”, “Responses”, or
“Comments”, the date/time of editing is not
shown in the post/ response/ comment. Only
the period of time elapsed since the creation
of the post/ response/ comment is shown.
So, there is no way of knowing if any post,
response, or comment is the original version,
or an edited version.
“Editing post/
response/
comment”
interactions.
1. Visibility of
System Status.
3
Copyright © Sebbe Isaac Kurian - Page 32
Issue
#
Description
System
Location
Heuristics
Violated
Severity
6. The “Insert Image” dialog box’s interface and
error messages use system-oriented
language that may confuse most of the
Users.
Also, if the User gives the URL of an image
file that does not exist, or an invalid URL,
there is no error or any other feedback.
“Insert Image”
pop-up Dialog
box.
1. Visibility of
System Status.
2. Match
between
System and the
Real World.
9. Help Users
Recognize,
Diagnose, and
Recover from
Errors.
3
7. While adding a “Post”, the “Topic area” field is
set as “General” by default. As the input field
is shown as already filled-up, the Users may
overlook it and skip it – causing the “Post” to
be submitted under the “General” topic area
by default.
“Add a Post”
screen.
1. Visibility of
System Status.
5. Error
Prevention.
2
8. The User can enter a whole page of text in
the “Title” field of a “Post”. If the “Post” is then
submitted, this entire title is displayed in the
post’s thumbnail view on the left panel. This
causes the thumbnails of previous posts to
scroll out of view.
“Add a Post”
and “Editing
post” screens.
5. Error
Prevention.
2
Copyright © Sebbe Isaac Kurian - Page 33
Issue
#
Description
System
Location
Heuristics
Violated
Severity
9. If the User clicks the “Submit” button for a
“Response” or “Comment”, without entering
any text in the corresponding Text Entry Box,
nothing happens. There is no error, or any
other feedback.
“Add a
response” and
“Add a
comment”
interactions.
1. Visibility of
System Status.
5. Error
Prevention.
9. Help Users
Recognize,
Diagnose, and
Recover from
Errors.
2
10. The collapsed “More” button in a published
“Comment” could overlap with the comment’s
content.
So, Users may not notice the button, if
overlapping happens.
This overlap issue in not there for “Posts” or
“Responses”.
Viewing a
“Post” with
“Response(s)”
and
“Comment(s)”.
1. Visibility of
System Status.
4. Consistency
and Standards.
2
11. Users can “Vote” for other User’s “Posts” and
“Responses”. The “Vote” button is displayed
while viewing their own “Posts” and
“Comments”, but does not work. This may
confuse some Users.
Viewing a
“Post” with
“Response(s)”.
1. Visibility of
System Status.
2
Copyright © Sebbe Isaac Kurian - Page 34
Issue
#
Description
System
Location
Heuristics
Violated
Severity
12. The clickable buttons used in the adding and
editing of “Posts”, “Responses”, and
“Comments” are not consistently styled.
For example, the “Submit” button for adding a
“Response” is white with blue text, which
becomes blue with white text on mouse
hover.
Meanwhile, the “Submit” buttons for adding a
“Post”, and for adding a “Comment” is blue
with white text, which becomes darker blue
with white text on mouse hover.
This inconsistent styling may confuse some
Users.
“Add a post/
response/
comment”,
and “Editing
post/
response/
comment”
interactions.
4. Consistency
and Standards.
2
13. The icon for inserting a hyperlink, shown in
the formatting toolbar of the Text Entry Box –
a globe with an arrow – is not a commonly
used representation. The “linked-chain” icon
is more universally used for this, which also
invokes the “link” metaphor. So, some Users
may not interpret the globe icon correctly.
“Add a post/
response/
comment”,
and “Editing
post/
response/
comment”
interactions.
2. Match
between
System and the
Real World.
4. Consistency
and Standards.
2
Copyright © Sebbe Isaac Kurian - Page 35
Issue
#
Description
System
Location
Heuristics
Violated
Severity
14. In the thumbnail view of the “Posts” on the left
panel, the title and the beginning of the first
line of each post is shown. This first line of
the post is displayed in the raw format with all
the formatting characters. This may make it
difficult for the Users to quickly scan it, if lots
of formatting has been applied in the
beginning.
The “Post”
thumbnails
view on the
left panel.
2. Match
between
System and the
Real World.
1
15. When adding a “Post”, there is a line – “This
post will be visible to everyone” – shown
between the input fields “Topic area” and
“Title”. The relevance and context of this
information at this particular position on the
screen is not clear. This may not make sense
to some Users.
“Add a Post”
screen.
8. Aesthetic
and Minimalist
Design.
1

More Related Content

PPT
User Research 101
PDF
Interaction design beyond human computer interaction
PDF
Usability and User Experience
PDF
UX & UI Design 101
PDF
Heuristic ux-evaluation
PDF
Usability testing
PDF
Simple Steps to UX/UI Web Design
PPTX
UX Design process, #UX, #Design Process, #Agile UX
User Research 101
Interaction design beyond human computer interaction
Usability and User Experience
UX & UI Design 101
Heuristic ux-evaluation
Usability testing
Simple Steps to UX/UI Web Design
UX Design process, #UX, #Design Process, #Agile UX

What's hot (20)

PDF
10 Usability Heuristics explained
PDF
Usability Testing 101 - an introduction
PDF
Basic Principles of Interface design
PDF
PDF
User Interface (UI) Design Ideation of Innovative Elevator Consoles
PDF
Introduction to Usability Testing: The DIY Approach - GA, London January 13th...
PPT
The design of everyday things
PPTX
The Importance of UX
PPTX
10 Usability Heuristics for User Interface Design
PDF
UI & UX Design for Startups
PPTX
Ux design-fundamentals
PDF
UX/UI Design 101
PPTX
UI/UX Fundamentals
PPTX
i/o extended: Intro to <UX> Design
PPTX
UI UX introduction
PDF
Introduction to Information Architecture
PDF
UX design
PPTX
UI UX Introductory session
PPTX
A UI and UX training presentation
PDF
UX/UI design
10 Usability Heuristics explained
Usability Testing 101 - an introduction
Basic Principles of Interface design
User Interface (UI) Design Ideation of Innovative Elevator Consoles
Introduction to Usability Testing: The DIY Approach - GA, London January 13th...
The design of everyday things
The Importance of UX
10 Usability Heuristics for User Interface Design
UI & UX Design for Startups
Ux design-fundamentals
UX/UI Design 101
UI/UX Fundamentals
i/o extended: Intro to <UX> Design
UI UX introduction
Introduction to Information Architecture
UX design
UI UX Introductory session
A UI and UX training presentation
UX/UI design
Ad

Similar to Heuristic Evaluation based on Nielsen's 10 Usability Heuristics (20)

PDF
Heuristic evaluation
PPT
Universal usability engineering
PDF
Why Usability Testing Should be Part of your Accessibility Testing Strategy
PDF
Heuristics Evaluation - How to Guide.pdf
PDF
Heuristics Evaluation - How to Guide.pdf
DOCX
eFolioMinnesota Text-Based Usability Test Findings and Analysis Report
PDF
Ijetr021224
PDF
Interaction Patterns In User Interfaces
PPTX
Accessibility, Usability and User Centred Design (Usabiltiy)
PPTX
Design process design rules
PPT
Master thesis presentation
PPT
Master thesis presentation
PPT
Ui Design And Usability For Everybody
PDF
2012 in tech-usability_of_interfaces (1)
PDF
UXPA 2023 Poster: 5 Key Findings from Moderated Accessibility Testing with Sc...
PDF
Principles of Health Informatics: Usability of medical software
PPT
Usability
PPTX
“Elemental elements”
Heuristic evaluation
Universal usability engineering
Why Usability Testing Should be Part of your Accessibility Testing Strategy
Heuristics Evaluation - How to Guide.pdf
Heuristics Evaluation - How to Guide.pdf
eFolioMinnesota Text-Based Usability Test Findings and Analysis Report
Ijetr021224
Interaction Patterns In User Interfaces
Accessibility, Usability and User Centred Design (Usabiltiy)
Design process design rules
Master thesis presentation
Master thesis presentation
Ui Design And Usability For Everybody
2012 in tech-usability_of_interfaces (1)
UXPA 2023 Poster: 5 Key Findings from Moderated Accessibility Testing with Sc...
Principles of Health Informatics: Usability of medical software
Usability
“Elemental elements”
Ad

Recently uploaded (20)

PDF
analisis snsistem etnga ahrfahfffffffffffffffffffff
PPT
EthicsNotesSTUDENTCOPYfghhnmncssssx sjsjsj
PDF
Test slideshare presentation for blog post
PPTX
a group casestudy on architectural aesthetic and beauty
PPTX
2. Competency Based Interviewing - September'16.pptx
PPTX
8086.pptx microprocessor and microcontroller
PDF
The Basics of Presentation Design eBook by VerdanaBold
PPTX
ENG4-Q2-W5-PPT (1).pptx nhdedhhehejjedheh
PPTX
Final Presentation of Reportttttttttttttttt
PDF
GSH-Vicky1-Complete-Plans on Housing.pdf
PDF
Social Media USAGE .............................................................
PPTX
22CDH01-V3-UNIT-I INTRODUCITON TO EXTENDED REALITY
PDF
Trends That Shape Graphic Design Services
PPTX
URBAN FINANCEnhynhynnnytnynnnynynyynynynyn
PPTX
Necrosgwjskdnbsjdmdndmkdndndnmdndndkdmdndkdkndmdmis.pptx
PPTX
22CDO02-IMGD-UNIT-I-MOBILE GAME DESIGN PROCESS
PPTX
UI UX Elective Course S1 Sistem Informasi RPS.pptx
PPTX
UNIT III - GRAPHICS AND AUDIO FOR MOBILE
PPTX
Presentation1.pptxnmnmnmnjhjhkjkjkkjkjjk
PPT
WHY_R12 Uaafafafpgradeaffafafafaffff.ppt
analisis snsistem etnga ahrfahfffffffffffffffffffff
EthicsNotesSTUDENTCOPYfghhnmncssssx sjsjsj
Test slideshare presentation for blog post
a group casestudy on architectural aesthetic and beauty
2. Competency Based Interviewing - September'16.pptx
8086.pptx microprocessor and microcontroller
The Basics of Presentation Design eBook by VerdanaBold
ENG4-Q2-W5-PPT (1).pptx nhdedhhehejjedheh
Final Presentation of Reportttttttttttttttt
GSH-Vicky1-Complete-Plans on Housing.pdf
Social Media USAGE .............................................................
22CDH01-V3-UNIT-I INTRODUCITON TO EXTENDED REALITY
Trends That Shape Graphic Design Services
URBAN FINANCEnhynhynnnytnynnnynynyynynynyn
Necrosgwjskdnbsjdmdndmkdndndnmdndndkdmdndkdkndmdmis.pptx
22CDO02-IMGD-UNIT-I-MOBILE GAME DESIGN PROCESS
UI UX Elective Course S1 Sistem Informasi RPS.pptx
UNIT III - GRAPHICS AND AUDIO FOR MOBILE
Presentation1.pptxnmnmnmnjhjhkjkjkkjkjjk
WHY_R12 Uaafafafpgradeaffafafafaffff.ppt

Heuristic Evaluation based on Nielsen's 10 Usability Heuristics

  • 1. Copyright © Sebbe Isaac Kurian Heuristic Evaluation of edX Website’s Discussion Forum (based on Nielsen's 10 Usability Heuristics) Heuristic Evaluation to identify usability issues in the edX website’s (www.edx.org) Discussion Forum – based on Jakob Nielsen's 10 Usability Heuristics for User Interface Design. Sebbe Isaac Kurian, January 2018.
  • 2. Copyright © Sebbe Isaac Kurian - Page 2 Contents Executive Summary 3 Introduction 4 Findings and Recommendations 6 Conclusion 26 Reference: Nielsen’s Usability Heuristics 28 Appendix: List of Usability Issues 30
  • 3. Copyright © Sebbe Isaac Kurian - Page 3 Executive Summary edX (www.edx.org) is a MOOC (Massive Open Online Course) provider, serving learners worldwide. Each course on edX has a Discussion Forum. This Heuristic Evaluation was done to determine the usability of edX's Discussion Forum. The Discussion Forum was evaluated against Nielsen's “10 Usability Heuristics for User Interface Design”. 15 usability issues were identified, and each of their severity assessed on a 1 to 4 scale. The 10 most severe of these usability issues were examined in detail. They are: 1. Absence of an in-place text editor for creating and editing Posts, Responses, and Comments. 2. Error handling of the mandatory input fields when adding a new Post is not optimal. 3. While entering text, cannot start a new line or paragraph by pressing the “Enter” key. 4. Cannot cancel the process of adding a Response or Comment, once started. 5. Date/Time of editing is not shown in the Posts, Responses, or Comments. 6. When inserting an image in the Posts, Responses, or Comments, the error handling is not optimal. 7. The process of setting a topic when adding a Post, is not optimal. 8. There is no optimum character limit set for the Title of a Post. 9. Error handling when submitting a new Response or Comment, is not optimal. 10. The More button in a published Comment may overlap with the Comment’s content. For each of these issues, suggestions to improve the usability are recommended. Lastly, the results are summarized, and the limitations of this evaluation clarified.
  • 4. Copyright © Sebbe Isaac Kurian - Page 4 Introduction About edX edX (www.edx.org) is a MOOC (Massive Open Online Course) provider, offering high-quality courses from top universities and institutions, to learners all over the world. Founded by Harvard University and MIT in 2012, edX is non-profit and open- source. Open edX is the open-source platform that powers edX courses. Each course on edX has a Discussion Forum associated with it. Being fully online, this Discussion Forum is important for an interactive student experience. On the Discussion Forum, the learners, course staff, and community teaching assistants interact with each other. They can browse and contribute (ask/answer/share) to course discussions. Why Evaluate? Efficient and effective learning are key goals of edX. As the core element facilitating interaction in a course, the usability of the Discussion Forum is crucial for efficient and effective learning. Heuristic Evaluation is a cost effective and easy-to-use UX research method, that can identify usability issues in a user interface. It is a usability inspection technique which involves evaluating the user interface against a set of recognized usability principles (also known as heuristics).
  • 5. Copyright © Sebbe Isaac Kurian - Page 5 Evaluation Procedure For this evaluation, the “10 Usability Heuristics for User Interface Design”, formulated by Jakob Nielsen (Nielsen 1994a) was used. They are: 1. Visibility of System Status. 2. Match between System and the Real World. 3. User Control and Freedom. 4. Consistency and Standards. 5. Error Prevention. 6. Recognition rather than Recall. 7. Flexibility and Efficiency of Use. 8. Aesthetic and Minimalist Design. 9. Help Users Recognize, Diagnose, and Recover from Errors. 10. Help and Documentation. An overview of these Heuristics is given in the Reference section of this document. 15 usability issues were identified. The evaluation determined usability issues in terms of, which heuristic(s) from the above list is(are) being violated, and the severity of violation. The severity was assessed according to the following scale: 1 = Cosmetic Problem. No real usability impact. (It is just annoying). 2 = Minor Usability Problem. Fix it, if there is time. 3 = Major Usability Problem. Important to fix. 4 = Usability Catastrophe. Must fix. (Prevents Users from completing the task). These 15 issues were compiled into a prioritized list, ranked in decreasing order of severity (Refer Appendix). The top 10 usability issues in this list were examined in detail. These, along with the suggestions for improvement for each, are given in the following section, Findings and Recommendations.
  • 6. Copyright © Sebbe Isaac Kurian - Page 6 Findings and Recommendations Finding-1 Absence of an in-place text editor for creating and editing Posts, Responses, and Comments. Severity: 3. Heuristics Violated: 4. Consistency and Standards. Description: This relates to the “Add a post/response/comment”, and “Editing post/response/comment” interactions. For adding and editing “Posts”, “Responses”, or “Comments” the Users enter/edit/format text in a Text Entry Box. The resulting formatted text that will be published, is displayed in a separate “Preview” box under it. Figure 1.1: Action and Result happening in Different Places.
  • 7. Copyright © Sebbe Isaac Kurian - Page 7 This violates Heuristic 4 - Consistency and Standards (Nielsen 1994a), according to which, conforming to universally followed conventions increase the usability of an interface – by taking advantage of the Users’ previous experience and existing knowledge. The common practice in most text editors is, in-place editing and formatting of text – where the action and result happens in the same place. Making the User switch attention between the separate areas – do the action in one place, then view the result in another place – increase the perceived complexity of the interface. Many Users would find this back-and-forth switching confusing. Suggestions for Improvement: Provide an in-place text editing interface, like in Windows WordPad, Mac TextEdit, MS-Word, etc. Figure 1.2: In-Place text editing in Windows WordPad.
  • 8. Copyright © Sebbe Isaac Kurian - Page 8 Finding-2 Error handling of the mandatory input fields when adding a new Post is not optimal. Severity: 3. Heuristics Violated: 1. Visibility of System Status. 5. Error Prevention. 9. Help Users Recognize, Diagnose, and Recover from Errors. Description: This pertains to the “Add a Post” screen. The field labelled “Title”, and the text box labelled “Your question or idea” are required to be filled by the User. The “(Required)” note appended to the labels, is not highlighted enough visually to get Users’ attention, and so could easily be overlooked. This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), according to which, the interface should support discoverability of the actions that Users are supposed to take, by informing them appropriately. Figure 2.1: Error Handling of required fields in the "Add a Post" screen.
  • 9. Copyright © Sebbe Isaac Kurian - Page 9 Users can skip either one or both fields without getting any feedback, until they click the “Submit” button. This again violates Heuristic 1 - Visibility of System Status, which advocates appropriate and timely feedback to Users. This also violates Heuristic 5 - Error Prevention (Nielsen 1994a), because the interface is not designed in such a way, so as to “prevent the problem from occurring in the first place” – here the required fields can be skipped without immediate warning. When error does occur, the Error Notification is not sufficiently highlighted, and does not effectively direct the Users’ attention to the place(s) where the error occurred. Also, if the User skips the text box, the Error Message says – “Body can’t be empty” – a language usage that some Users may not understand. This violates Heuristic 9 - Help Users Recognize, Diagnose, and Recover from Errors (Nielsen 1994a), according to which, “error messages should be expressed in plain language”, and should help the Users by clearly indicating why, where, and how to solve it. Suggestions for Improvement: The required fields should be indicated more strikingly. There should be in-process checking and feedback, to prevent errors before the “Submit” button is clicked. The Error Notification should help the Users to solve the problem easily. An example of this is the “Create an account” interaction in the www.facebook.com landing page. Figure 2.2: The "Create an account" interaction of Facebook.
  • 10. Copyright © Sebbe Isaac Kurian - Page 10 Finding-3 While entering text, cannot start a new line or paragraph by pressing the “Enter” key. Severity: 3. Heuristics Violated: 4. Consistency and Standards. 10. Help and Documentation. Description: In the text entry box used for adding and editing Posts, Responses, and Comments, it is not possible for Users to start a new line or new paragraph, by pressing the “Enter” key. Pressing the “Enter” key gives a blank space. Figure 3.1: Pressing "Enter" key in the text box. This behavior violates Heuristic 4 - Consistency and Standards (Nielsen 1994a), because the common standard practice of pressing the “Enter” key for a new line/paragraph is not followed.
  • 11. Copyright © Sebbe Isaac Kurian - Page 11 On the other hand, pressing the “Enter” key twice in a row, starts a new paragraph. And for starting a new line, the User must press the “Space” bar twice followed by the “Enter” key. These are not standard behavior in text editors – again a violation of Heuristic 4 - Consistency and Standards. Also, these key combinations are not communicated to the Users – a violation of Heuristic 10 - Help and Documentation (Nielsen 1994a), which advocates providing help focused on the User’s task. Suggestions for Improvement: It would be ideal to support the standard text-entry practice, of pressing the “Enter” key once for a new line/paragraph. If for any specific reason, uncommon key combinations are required for this, that information should be communicated to the User via a help option. For example, like the help provided in the Notepad++ editor. Figure 3.2: Help option in the Notepad++ editor.
  • 12. Copyright © Sebbe Isaac Kurian - Page 12 Finding-4 Cannot cancel the process of adding a Response or Comment, once started. Severity: 3. Heuristics Violated: 3. User Control and Freedom. 4. Consistency and Standards. Description: When Users are viewing a “Post”, the can give a “Response” to it, as well as “Comment” on any “Response” already given. If the Users click to add a Response or Comment, and then decide not to save and publish it – they cannot cancel it. There is only a “Submit” button for saving and publishing Responses and Comments – No “Cancel” button is provided. This is a violation of Heuristic 3 - User Control and Freedom (Nielsen 1994a), which advocates the need to provide a clearly marked “Emergency Exit” to leave an unwanted state, and support Undo. Figure 4.1: There is no "Cancel" button when adding a Response or Comment. On the other hand, while adding a Post (in the “Add a Post” screen), there is a “Cancel” button along with the “Submit” button. So, not providing a “Cancel” button for Responses and Comments, also violates Heuristic 4 - Consistency and Standards (Nielsen 1994a), because the interface is not consistent within the system, and may confuse some Users.
  • 13. Copyright © Sebbe Isaac Kurian - Page 13 Figure 4.2: There is a "Cancel" button when adding a Post. Suggestions for Improvement: Provide a “Cancel” button when adding Responses and Comments – like the one given when adding a Post. This would give Users the freedom to exit the process if so desired.
  • 14. Copyright © Sebbe Isaac Kurian - Page 14 Finding-5 Date/Time of editing is not shown in the Posts, Responses, or Comments. Severity: 3. Heuristics Violated: 1. Visibility of System Status. Description: When Users submit a new Post, Response to a Post, or Comment to a Response, the duration of time elapsed since submission is displayed with the post/response/comment – in minutes, hours, days, months, etc. Figure 5.1: Time elapsed since submission of a Post, Response, and Comment.
  • 15. Copyright © Sebbe Isaac Kurian - Page 15 If Users edit their post/response/comment later on, this displayed time period is not updated to show the time when editing was done. So, there is no way of knowing if any Post, Response, or Comment is the original version, or an edited version. This could compromise the integrity of the information displayed, when a Post having Response(s), or a Response with Comment(s) is edited. Because, then the Response(s) and Comment(s) will pertain to the respective original Post or Response, and not to the edited content – and the Users who are viewing/reading it will not know about this. This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), according to which, the Users should be informed about the current system state – which in this case, is the time of editing of the post/response/comment. Suggestions for Improvement: Ideally, both the date and time of submission (not the duration of time elapsed since submission), and the date and time of editing (if done), should be given in the Posts, Responses, and Comments.
  • 16. Copyright © Sebbe Isaac Kurian - Page 16 Finding-6 When inserting an image in the Posts, Responses, or Comments, the error handling is not optimal. Severity: 3. Heuristics Violated: 1. Visibility of System Status. 2. Match between System and the Real World. 5. Error Prevention. 9. Help Users Recognize, Diagnose, and Recover from Errors. Description: When adding or editing a Post, Response, or Comment, Users can insert images in it. Clicking the button for inserting image in the tool bar of the text box (or pressing Ctrl-G), pops-up an “Insert Image” dialog box. Users can upload an image from their computer, or give the URL of an image. Figure 6.1: Dialog box for inserting Image.
  • 17. Copyright © Sebbe Isaac Kurian - Page 17 The two different use cases were evaluated – uploading an image, and giving the URL of an image. 1. Trying to upload a big sized image file – gives a terse pop-up error message. Figure 6.2: Error message pop-up on trying to upload a big image file. This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), because the interface does not inform the Users beforehand, about the maximum file size limit. This violates Heuristic 5 - Error Prevention (Nielsen 1994a), because the system does not try to prevent this problem from occurring, before initiating the upload. This violates Heuristic 2 - Match between System and the Real World (Nielsen 1994a), because the error message uses system-oriented terms, instead of communicating the problem in User understandable language. This violates Heuristic 9 - Help Users Recognize, Diagnose, and Recover from Errors (Nielsen 1994a), because the error message is not in plain language, and does not suggest a constructive solution for the Users. 2. Entering an invalid URL of an image file that does not exist – gives no error. This violates Heuristic 1 - Visibility of System Status, because the interface does not inform the User about what is happening, through appropriate feedback. This violates Heuristic 9 - Help Users Recognize, Diagnose, and Recover from Errors, because the interface does not give any error – so, the Users cannot recognize, or diagnose, or recover from this error.
  • 18. Copyright © Sebbe Isaac Kurian - Page 18 Suggestions for Improvement: Inform the Users beforehand, about the file size limit for uploads. Check the file size on file selection, and notify the User in case of exceeding the size limit, before initiating the upload process. If the User enters an image URL, check its availability, and inform the User about any problems. In both the above cases, give enough information in plain language, so that Users can understand what is happening, and know what to do. Figure 6.3, shows an example of a good error message “expressed in plain language”, that “precisely indicate the problem”, and “constructively suggest a solution”. Figure 6.3: An example of a good error message.
  • 19. Copyright © Sebbe Isaac Kurian - Page 19 Finding-7 The process of setting a topic when adding a Post, is not optimal. Severity: 2. Heuristics Violated: 1. Visibility of System Status. 5. Error Prevention. Description: In the “Add a Post” screen, the “Topic area” field of the Post is set as “General” by default. As the input field is shown as already filled-up, the User may not pay attention to it and overlook it – causing the Post to be submitted under the “General” topic area by default. This may not be what the User intended. Figure 7.1: Topic area of a Post set as "General" by default.
  • 20. Copyright © Sebbe Isaac Kurian - Page 20 This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), because it conveys potentially misleading information about the system to the User – seeing an input field as already filled, the User who is scanning the interface may assume that there is nothing more to do, and skip it. This also violates Heuristic 5 - Error Prevention (Nielsen 1994a), because instead of preventing a problem from occurring in the first place, this could cause an error to happen. Suggestions for Improvement: Use a dummy placeholder like “Select a Topic”, to indicate the drop-down list of available Topic areas. This will prevent the Post from being submitted under an unintended Topic area. Also, follow the Suggestions for Improvement given for Finding-2, on how to effectively present the required input fields. This will prevent the Users from skipping the “Topic area” field.
  • 21. Copyright © Sebbe Isaac Kurian - Page 21 Finding-8 There is no optimum character limit set for the Title of a Post. Severity: 2. Heuristics Violated: 5. Error Prevention. Description: When adding or editing a Post, the User can enter a whole page of characters in the “Title” field of a Post. If the Post is then submitted/updated, this full title is displayed in the Post’s thumbnail view on the left panel. This causes all previous Post thumbnails to scroll out of view. Figure 8.1: Abnormally long Title in the Post's thumbnail view. This violates Heuristic 5 - Error Prevention (Nielsen 1994a), because the interface does not prevent this problem from occurring in the first place. Suggestions for Improvement: Set an optimum character limit for the Title, inform the Users about this limit, and prevent them from exceeding it. Long Title
  • 22. Copyright © Sebbe Isaac Kurian - Page 22 Finding-9 Error handling when submitting a new Response or Comment, is not optimal. Severity: 2. Heuristics Violated: 1. Visibility of System Status. 5. Error Prevention. 9. Help Users Recognize, Diagnose, and Recover from Errors. Description: If the Users submit a new Response or Comment, without any content in the text box, nothing happens – No feedback, No error. Figure 9.1: Adding a new Response or Comment without content.
  • 23. Copyright © Sebbe Isaac Kurian - Page 23 This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), because the interface does not give any feedback to the Users, informing them about what is happening. This violates Heuristic 5 - Error Prevention (Nielsen 1994a), because the interface does nothing to prevent this problem from occurring. This violates Heuristic 9 - Help Users Recognize, Diagnose, and Recover from Errors (Nielsen 1994a), because the interface does not give an error message – so, the Users cannot recognize, or diagnose, or recover from this error. Suggestions for Improvement: Follow the Suggestions for Improvement given for Finding-2, on how to effectively present the required input fields. This would clearly indicate to the Users that the text box should not be blank. Also, follow the Suggestions for Improvement given for Finding-4, by providing a “Cancel” button. This would give Users the freedom to exit the process if so desired.
  • 24. Copyright © Sebbe Isaac Kurian - Page 24 Finding-10 The More button in a published Comment may overlap with the Comment’s content. Severity: 2. Heuristics Violated: 1. Visibility of System Status. 4. Consistency and Standards. Description: When viewing a published Post with Response(s) and their Comment(s), the collapsed “More” button in a Comment may overlap with the Comment’s content. If this happens, the Users may not notice this button at all. The content does not overlap the button in Posts or Responses – this happens only in Comments. Figure 10.1: Collapsed More buttons in Post, Response, and Comment.
  • 25. Copyright © Sebbe Isaac Kurian - Page 25 This violates Heuristic 1 - Visibility of System Status (Nielsen 1994a), because the interface does not clearly inform the Users about available actions, when the button and content overlap. This also violates Heuristic 4 - Consistency and Standards (Nielsen 1994a), because this is an instance of inconsistency within the system. This possibility of overlapping is not there in Posts or Responses, as illustrated in Figure 10.1. This could happen only in Comments. Suggestions for Improvement: Keep the content of the Comment clear from its collapsed “More” button – like it is being done in Posts and Responses (as shown in Figure 10.1).
  • 26. Copyright © Sebbe Isaac Kurian - Page 26 Conclusion The Heuristic Evaluation uncovered quite a few usability issues in the edX website's Discussion Forum. The most severe of them are: 1. Absence of an in-place text editor for creating and editing Posts, Responses, and Comments. 2. Error handling of the mandatory input fields when adding a new Post is not optimal. 3. While entering text, cannot start a new line or paragraph by pressing the “Enter” key. 4. Cannot cancel the process of adding a Response or Comment, once started. 5. Date/Time of editing is not shown in the Posts, Responses, or Comments. 6. When inserting an image in the Posts, Responses, or Comments, the error handling is not optimal. 7. The process of setting a topic when adding a Post, is not optimal. 8. There is no optimum character limit set for the Title of a Post. 9. Error handling when submitting a new Response or Comment, is not optimal. 10. The More button in a published Comment may overlap with the Comment’s content. Suggestions for improvement have been recommended to address these interface problems. By implementing these suggestions, the usability of the Discussion Forum would improve. This should significantly enhance the learning experience of students, and in turn help edX meet the key goals of efficient and effective learning.
  • 27. Copyright © Sebbe Isaac Kurian - Page 27 Limitations This Heuristic Evaluation was done by a single evaluator. According to Jakob Nielsen, single evaluators find on an average only 35 percent of the usability problems in interfaces. (Refer: “How to Conduct a Heuristic Evaluation” by Jakob Nielsen, January 1, 1995. URL: https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation). He points out that, since different evaluators tend to find different problems, better results are achieved by combining the evaluations from several evaluators. Considering the ratio of the benefits to the costs, he recommends 3 to 5 evaluators for the best, cost effective results.
  • 28. Copyright © Sebbe Isaac Kurian - Page 28 Reference 10 Usability Heuristics for User Interface Design Nielsen, J. (1994a). Enhancing the explanatory power of usability heuristics. Proc. ACM CHI'94 Conf. (Boston, MA, April 24-28), 152-158. Retrieved from https://www.nngroup.com/articles/ten-usability-heuristics/ Copyright © 2005 by Jakob Nielsen. ISSN 1548-5552 Jakob Nielsen's 10 general principles for interaction design are called "heuristics" because they are broad rules of thumb and not specific usability guidelines. 1. Visibility of System Status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. 2. Match between System and the Real World The system should speak the users' language, with words, phrases, and concepts familiar to the user, rather than system-oriented terms. Follow real- world conventions, making information appear in a natural and logical order. 3. User Control and Freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo. 4. Consistency and Standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
  • 29. Copyright © Sebbe Isaac Kurian - Page 29 5. Error Prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action. 6. Recognition rather than Recall Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate. 7. Flexibility and Efficiency of Use Accelerators — unseen by the novice user — may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions. 8. Aesthetic and Minimalist Design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. 9. Help Users Recognize, Diagnose, and Recover from Errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution. 10. Help and Documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
  • 30. Copyright © Sebbe Isaac Kurian - Page 30 Appendix: List of Usability Issues Table 1: List of Usability Issues found in the edX website’s Discussion Board. Issue # Description System Location Heuristics Violated Severity 1. For adding and editing “Posts”, “Responses”, or “Comments” the Users enter/edit/format text in a Text Entry Box. The resulting formatted text that will be published, is displayed in a separate “Preview” box under it. This could be confusing for many Users, as the common practice in most text editors is that, the result of editing and formatting is visible where it is done. “Add a post/ response/ comment”, and “Editing post/ response/ comment” interactions. 4. Consistency and Standards. 3 2. Error handling of the input field labelled “Title”, and the Text Entry Box labelled “Your question or idea” is not optimal. Both are “required” fields that cannot be left blank. The “(Required)” note appended to the labels is not highlighted enough to get Users’ attention, and so could easily be overlooked. Users can skip either one or both of these fields without getting any feedback, until they click the “Submit” button. Also, the error messages are not sufficiently highlighted, and does not effectively direct the Users’ attention to the place(s) where the error occurred. “Add a Post” screen. 1. Visibility of System Status. 5. Error Prevention. 9. Help Users Recognize, Diagnose, and Recover from Errors. 3
  • 31. Copyright © Sebbe Isaac Kurian - Page 31 Issue # Description System Location Heuristics Violated Severity 3. In the Text Entry Box, it is not possible for Users to start a new line, or new paragraph, using the common practice of pressing the “Enter” key. Pressing the “Enter” key once, gives a blank space. Pressing the “Enter” key twice in a row starts a new paragraph. This is not common practice in text entry. Also, this feature is not communicated to the User. “Add a post/ response/ comment”, and “Editing post/ response/ comment” interactions. 4. Consistency and Standards. 10. Help and Documentation. 3 4. There is a “Submit” button for saving and publishing “Responses” and “Comments”, but there is no “Cancel” button. So, if Users click to add a “Response” or “Comment”, and then decide not to proceed, they are stuck. They cannot undo. Meanwhile, for “Posts”, both “Submit” and “Cancel” buttons are provided. “Add a response” and “Add a comment” interactions. 3. User Control and Freedom. 4. Consistency and Standards. 3 5. If Users edit their “Posts”, “Responses”, or “Comments”, the date/time of editing is not shown in the post/ response/ comment. Only the period of time elapsed since the creation of the post/ response/ comment is shown. So, there is no way of knowing if any post, response, or comment is the original version, or an edited version. “Editing post/ response/ comment” interactions. 1. Visibility of System Status. 3
  • 32. Copyright © Sebbe Isaac Kurian - Page 32 Issue # Description System Location Heuristics Violated Severity 6. The “Insert Image” dialog box’s interface and error messages use system-oriented language that may confuse most of the Users. Also, if the User gives the URL of an image file that does not exist, or an invalid URL, there is no error or any other feedback. “Insert Image” pop-up Dialog box. 1. Visibility of System Status. 2. Match between System and the Real World. 9. Help Users Recognize, Diagnose, and Recover from Errors. 3 7. While adding a “Post”, the “Topic area” field is set as “General” by default. As the input field is shown as already filled-up, the Users may overlook it and skip it – causing the “Post” to be submitted under the “General” topic area by default. “Add a Post” screen. 1. Visibility of System Status. 5. Error Prevention. 2 8. The User can enter a whole page of text in the “Title” field of a “Post”. If the “Post” is then submitted, this entire title is displayed in the post’s thumbnail view on the left panel. This causes the thumbnails of previous posts to scroll out of view. “Add a Post” and “Editing post” screens. 5. Error Prevention. 2
  • 33. Copyright © Sebbe Isaac Kurian - Page 33 Issue # Description System Location Heuristics Violated Severity 9. If the User clicks the “Submit” button for a “Response” or “Comment”, without entering any text in the corresponding Text Entry Box, nothing happens. There is no error, or any other feedback. “Add a response” and “Add a comment” interactions. 1. Visibility of System Status. 5. Error Prevention. 9. Help Users Recognize, Diagnose, and Recover from Errors. 2 10. The collapsed “More” button in a published “Comment” could overlap with the comment’s content. So, Users may not notice the button, if overlapping happens. This overlap issue in not there for “Posts” or “Responses”. Viewing a “Post” with “Response(s)” and “Comment(s)”. 1. Visibility of System Status. 4. Consistency and Standards. 2 11. Users can “Vote” for other User’s “Posts” and “Responses”. The “Vote” button is displayed while viewing their own “Posts” and “Comments”, but does not work. This may confuse some Users. Viewing a “Post” with “Response(s)”. 1. Visibility of System Status. 2
  • 34. Copyright © Sebbe Isaac Kurian - Page 34 Issue # Description System Location Heuristics Violated Severity 12. The clickable buttons used in the adding and editing of “Posts”, “Responses”, and “Comments” are not consistently styled. For example, the “Submit” button for adding a “Response” is white with blue text, which becomes blue with white text on mouse hover. Meanwhile, the “Submit” buttons for adding a “Post”, and for adding a “Comment” is blue with white text, which becomes darker blue with white text on mouse hover. This inconsistent styling may confuse some Users. “Add a post/ response/ comment”, and “Editing post/ response/ comment” interactions. 4. Consistency and Standards. 2 13. The icon for inserting a hyperlink, shown in the formatting toolbar of the Text Entry Box – a globe with an arrow – is not a commonly used representation. The “linked-chain” icon is more universally used for this, which also invokes the “link” metaphor. So, some Users may not interpret the globe icon correctly. “Add a post/ response/ comment”, and “Editing post/ response/ comment” interactions. 2. Match between System and the Real World. 4. Consistency and Standards. 2
  • 35. Copyright © Sebbe Isaac Kurian - Page 35 Issue # Description System Location Heuristics Violated Severity 14. In the thumbnail view of the “Posts” on the left panel, the title and the beginning of the first line of each post is shown. This first line of the post is displayed in the raw format with all the formatting characters. This may make it difficult for the Users to quickly scan it, if lots of formatting has been applied in the beginning. The “Post” thumbnails view on the left panel. 2. Match between System and the Real World. 1 15. When adding a “Post”, there is a line – “This post will be visible to everyone” – shown between the input fields “Topic area” and “Title”. The relevance and context of this information at this particular position on the screen is not clear. This may not make sense to some Users. “Add a Post” screen. 8. Aesthetic and Minimalist Design. 1