![Assessing the Quality of Web Sites
Layla Hasan1
and Emad Abuelrub2
1
Department of Information Science, Loughborough University, UK
2
Department of Computer Science, Zarqa Private University, Jordan
-
Abstract: In the age of information technology and with the rapid growth of the World Wide Web, the
spectacular growth trend in e-business, that has been experienced so far, is expected to continue. Since the
introduction of the first commercial website in 1994, electronic commerce has spread across the globe as a
marketing, sales, and communication phenomenon, changing the fact of some business sectors. Companies
seeking to achieve significant benefits through e-business need to create an effective and usable web presence to
ensure successful interaction and communication with their employees, partners, and customers. Web services
are being widely deployed throughout business, education, government, and other organizations. The Internet
creates a new business environment, far different from anything that has come before; because it lets a company
conducts its entire set of business processes and practices online. As the dependency on web technology
increases, the need to assess the factors associated with website success increases as well. This paper reviewed
the most recent evaluation criteria methods, which were used in different e-business services. Furthermore, it
proposes general criteria for evaluating the quality of any website regardless of the type of service that it offers.
The dimensions of the criteria are content quality, design quality, organization quality, and user-friendly quality.
These dimensions together with their comprehensive indicators and checklist can be used by web designers and
developers to create quality websites to improve the electronic service and then the image of any organization
on the Internet.
Keywords: e-commerce, criteria, framework, website evaluation, website quality.
(Received May 29, 2008 / Accepted October 29, 2008)
1. Introduction
Since the introduction of the first electronic web
service in the mid nineties of the last century,
electronic services have spread across the globe in
various shapes, changing the face of many business
sectors. The new e-revolution is not only driving the
global economics but also transforming societies into
a knowledge-based economics all over the world. In
the last few years, the Internet has shown a rapid
growth in term of commercial trade volume, which
led to a new definition of almost all aspects of
business. The deployment of recent information and
communication technologies enabled firms to provide
higher quality services, lower prices for customers,
and increase the profit margins for businesses. Also,
the new technology created new trends in business
and produced new era in business, finance, and
economics [21, 42].
The Internet created a new business environment,
far different from anything that has come before,
enabling any company to conduct its entire set of
business processes and practices online [45]. E-
business is any business process performed via an
Internet-based, computer-mediated network [34].
There are many categories of e-business; for example
Business to Business (B2B), Business to Consumer
(B2C), Consumer to Business (C2B), and Consumer
to Consumer (C2C). E-business and e-commerce tend
to be used interchangeably leading to policy
incoherence.
Recently, we have seen a proliferation of
electronic websites with a tremendous amount of
information either with high quality, or with low
quality, as well as sites that are outright misleading
[13, 18]. The number of websites grew from 17
million in the middle of 2000 to 65 million in the
middle of 2005 [37]. The explosion of the web has
determined the need of measurement criteria to
evaluate the aspects related to the quality in use, such
as usability and accessibility of a web application.
The objective is to make a website useful, profitable,
user linking, and accessible [38]. Awareness of
quality issues has recently affected every industrial
sector [32]. An organization with a website that is
difficult to use and interact with, gives a poor image
on the Internet and weaken an organization’s position.
Therefore, it is important for any organization to have
the ability to make an assessment of the quality of
their e-commerce service, in order to improve their
offerings over time and benchmark against
competitors and best practices in any industry [5].
In the last decade, numerous studies have focused
on the designs of websites for general information
seeking purposes and for electronic commerce
purposes [49]. The design and commercial
development of websites are very critical to e-
commerce success [5]. Numerous practitioner reports
l.hasan2@lboro.ac.uk abuelrub@zpu.edu.jo](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-2-320.jpg)
![and reviews have been published seeking to identify
the good and bad features of websites. Site reviews
range from theoretical and scientific opinions to
surveys of successful sites and features to existing
successful e-businesses. Although, there has been a
significant research on supporting electronic
commerce, most of the existing empirical research
focusing on success factors of websites is mainly
exploratory in nature [28]. Up to our knowledge, there
is no standard framework or benchmark defining
website effectiveness [6].
This paper reviewed the most recent evaluation
criteria methods which were used in different
websites, and proposes a general comprehensive
framework for evaluating the quality of any web
service regardless of the type of service that it offers.
The remaining of the paper is as follows. Section 2
gives a brief review of previous work. Section 3
discusses and analyzes the proposed framework.
Section 4 defines the dimensions of the proposed
framework and its indicators. Section 5 concludes the
paper and suggests some future work.
2. Previous Work
As the dependency on web services increases, the
need to assess characteristics with website quality and
success increases. Websites characteristics are
important; they have been a constant concern of
research in different domains and they were widely
studied in the e-commerce literature [17]. Although
there has been a significant research on supporting e-
commerce, many existing empirical studies focusing
on the quality of websites is mainly exploratory in
nature. Most of the current studies are either dealing
with a limited number of quality factors or directed
toward a specific web service. Thus, while there
should be a considerable number and variety of
factors associated with web site success, little research
exists about the combination of these factors and
services. Recently, research and studies are
accumulating including different models and/or
frameworks to evaluate the quality and performance
of websites. We categorized the previous studies that
investigated the quality dimensions of websites
according to the type of service that the website
offers; for example business and commercial,
educational, banking, governmental, and others. This
section briefly reviews the previous studies according
to the website service.
Business and commercial websites were studied
from different perspectives. Some researchers
investigated website features or factors that are
critical to e-business success, in which they called
them critical success factors [10, 27, 28, 30, 33].
Other researchers address key issues, ideas and
strategies to be considered in the management of
online business from customer satisfaction
perspective, and they assess whether a website has
been built with a customer’s goals in mind [8, 19, 41,
46, 49]. Another group of researchers investigated the
perspective of web designers in order to elicit factors
that they consider important when designing or
developing effective websites [8, 43]. Other
researchers developed generic tools or measurement
frameworks for the assessment of website quality [4,
5, 12, 14, 26, 32, 38]. Some researchers concentrated
on some important features; they either proposed a
framework to measure the important features of the
website or used previous models to find out to which
extent e-business websites incorporate these important
features. Others [6, 25, 40] concentrated in their
studies on website usability, Heimlich and Wang [19]
proposed key issues of website’s structure, while Cao
and Zhang [7] examined factors that affect e-
commerce website design. Heimlich [18] discussed
content evaluation of websites, Hussin et al. [20]
studied the extent in which companies incorporate
ethical and trustworthy elements on their websites,
while Fogg et al. [13] investigated how different
elements of websites affect people’s perception of
credibility.
Educational websites were also studied from
many different perspectives. Zhang and Dran [49]
developed a theoretical framework for evaluating
website quality from a user satisfaction perspective.
Others concentrated on some specific features of
websites. For example Lautenbach et al. [24]
developed a framework to measure usability of
websites, while Yoo and Jin [48] investigated and
evaluated the design of university websites. Other
researchers, while assessing the university websites,
took in consideration other features. Osborne and
Rinalducci [35] designed a criteria to evaluate web
resources for utilization within the context of
scholarly research within the discipline of the art
history. Singh and Sook [39] attempted to find
solutions to user problems and involved evaluating
South African university websites on certain factors.
Banking websites were studied from many
different perspectives using different models. Diniz et
al. [11] proposed a model to evaluate and build digital
business environment from the user’s point of view.
While other researchers proposed a specific
framework to evaluate the Internet banking websites
and the service quality of Internet banking [2, 44, 47].
Other researchers adopted through their studies a
number of previous models to evaluate Internet
banking websites. For example, Awamleh and
Fernandes [3] used Diniz Model to evaluate websites
of foreign and local banks in the United Arab
Emirates. Guru et al. [16] evaluated the web presence
of banks in Islamic countries based on a Diniz Model
too. Paynter and Chung [36] examined how New
Zealand banks enhanced their retail banking services
through the Internet using Herey’s Model for website
evaluation.
Governmental websites were studied from
different perspectives. Zhang and Dran [49]
developed a theoretical framework for evaluating
website quality from a user satisfaction perspective](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-3-320.jpg)
![through theoretical and empirical investigations.
While Krauss [23] identified seventeen
comprehensive quality dimensions that can be used to
rate website quality features that are important to e-
government websites. Others researchers concentrated
on other features. For example Kokkinaki et al. [22]
presented a framework for evaluating existing e-
government initiatives in Cyprus. It includes content
characteristics, design characteristics, and common
features of e-government websites. Choudrie et al. [9]
described the issues related to the accessibility,
quality, and privacy of government web using a
common set of performance metrics and web
diagnostic tools which are WebXact, Ntmechanic, and
Vizcheck. Ma and Zaphiris [29] studied the usability
and content accessibility of UK e-government
websites and investigated whether they are ranked
high in terms of accessibility. The usability and
accessibility of fifty selected UK e-government
websites were measured using two automatic
evaluation tools, Bobby and LIFT. Abanumy et al. [1]
investigated the issue that makes a website accessible
and explored the importance placed on web
accessibility with respect to e-government websites.
Other researchers addressed other kinds of web
services from different perspectives. Lin and Joyce
[27] studied different e-commerce models of online
auction websites. Six critical success factors for a
successful online auction website were identified.
They include design and content, consumer education,
security, customer support, online community, and
market positioning. Barnes and Vidgen [4] deployed
WEBQUAL in the domain of Internet auctions and
identified three quality dimensions; information,
interaction, and site design. Lim [25] evaluated the
impact of four relevant factors to e-shopping;
usefulness, ease of use, enjoyment, and security.
Results showed that perceived ease of use and
usefulness of the e-shopping website have a
significant direct impact on the success of the site.
3. Discussion and Analysis
The objective of this research is to develop a
theoretical, comprehensive, and measurable
framework for assessing the quality of websites in
order to provide straight forward criteria to encourage
improvements of website design and its
implementation. Furthermore, we aim to develop a
framework that is capable of reliable applications
across a broad range of websites regardless of the
service they provide. A multi-phase approach was
adopted that included a wide range of literature
review, review of leading sites, identification of
success factors from research and industry literature,
comparison of factors with published industry
scoring studies, and using our own experience in the
field. Our process overlaid industry and academic
research to identify quality factors in order to meet the
objectives of this research.
After deep and comprehensive review of different
evaluation methods and their elements that were used
in different services over the Internet, we propose 4-
dimensions criteria which is comprehensive and
includes all previous dimensions and elements, in
order to be used as general criteria to evaluate all
kinds of websites. The dimensions of the proposed
criteria are content quality, design quality,
organization quality, and user-friendly quality. In
order to investigate how our proposed criteria were
used in previous studies, we re-arranged each element
of each dimension of the previous work to be under
one of the four new dimensions. The result of re-
arranging the dimensions of previous work into the
proposed 4-dimensions criteria is shown in Figure 1.
Content
Quality; 37;
29%
Design
Quality; 30;
23%
Organization
Quality; 22;
17%
Usr-Friendly
Quality; 40;
31% Content Quality
Design Quality
Organization Quality
Usr-Friendly Quality
Figure 1. Common dimensions after re-arrangement.
We summarized the common quality dimensions
according to the service that a website offers.
Common dimensions that were used in evaluating e-
business and e-commerce websites were
concentrating on currency, accuracy, comprehensive
and value added content, ease of use, reliability,
availability of the needed information, speed of
downloading, customization, effective internal search,
different types of service and support to customers,
security and privacy in all types of transactions,
logical grouping of website elements, and attractive
design that will attract the user and encourages
him/her to spend more time in the website [4-7, 10,
12-14, 18-20, 25-28, 30, 32, 33, 38, 40, 41, 43, 46,
49].
Common dimensions that were used to assess the
quality of educational websites were concentrated on
currency, accuracy and comprehensibility of
information, ease of use, clear layout of websites, and
an attractive design [24, 35, 39, 48, 49]. Common
dimensions that were used to assess the quality of
banking websites focused on ease of use,
customization, internal search function, security of
transactions, aesthetic effects, and useful interaction
between users and websites in order to get instant
feedback from website to user [2, 3, 11, 16, 36, 44,
47, 49]. Common dimensions that were used to assess
the quality of governmental websites are quick
response time, up-to-date, accurate information,
effective search tool, easy to understand, and secure
transactions [1, 9, 22, 23, 29, 49]. Common](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-4-320.jpg)
![dimensions that were used to assess the quality of
auctions and e-shopping websites are design, content,
security, support, and ease of use [4, 25, 27].
4. The Proposed Framework
The proposed framework attempts to integrate
knowledge and experience from disparate sources, a
range of reference disciplines and empirical practices.
The objective is to identify measurable features and
indicators that currently comprise a successful web
site. A set of features is developed that comprise a
current representation of a perfect website. The
proposed framework can be used to compare between
the quality of websites, to identify a path for
improvement of a website, and to provide a guideline
for designers and developers when creating new
websites.
After we reviewed each evaluation criteria, we
added its indicators to the suitable place of the
proposed 4-dimension criteria, besides adding some
indicators in which we see them important from our
own experience. Our criteria include all main
indicators of the previous studies of evaluating the
quality of websites. Figure 2 summarizes the
hierarchy of the proposed framework.
Figure 2. Hierarchy of the proposed framework.
4.1 Content Quality
It is generally agreed that content quality is an
important dimension, which deals with the
characteristics of websites’ information. Singh and
Sook [39] called this dimension the king dimension of
any website, since it is the major source of value to
customers. This dimension has been addresses by a
variety of researchers in different ways. Some
researchers studied the content of websites without
taking into consideration other dimensions [15, 18],
while others considered content quality or information
quality as one of the basic dimensions of their
evaluating models [2, 4-6, 10, 22, 27, 28, 32, 33, 36,
38-40, 43].
The following indicators and check elements,
which are summarized in Table 1, are the most
important relating to the content quality dimension.
1. Timely: The currency of websites’ information
and how much it is up-to-date, how frequently the
website is updated, and is it clear when the site
was updated [4, 5, 7, 9, 12, 13, 15, 18, 19, 22, 24,
28, 30, 31, 33, 35, 38, 41, 49].
2. Relevant: The extent to which websites’
information is comprehensive, complete and
provides the right level of details [5, 10, 12, 14,
15, 18, 32, 33, 35, 43, 49]. The extent to which it
is informative, meaning, value added according
to its audience [22, 46], and fits to users’ need [7,
22, 23, 32, 35, 39]. So, websites include
information about the organization’s objectives
[3, 19, 22, 36], organization’s history [3, 6],
customers or audience [15, 18, 36], products or
services [6, 28, 36], and photographs of
organization’s facilities to reduce customer fears
of dealing with the website [6].
3. Multilanguage/Culture: The websites’
information is available in different languages [1,
12, 13, 22, 23, 26, 44], suitable to different
cultures [12, 23], and meets the needs of all
customers regardless of their county.
4. Variety of presentation: Information is presented
in different forms (text (.doc, .pdf, … ), video,
audio, … ), so that the user can download the
form that suits him/her [22, 39, 44].
5. Accuracy: Information is precise, there is no
spelling errors or grammar errors [4, 5, 7, 12, 14,
15, 18, 22, 23, 28, 32, 33, 39, 41, 49] and the
sources of information is identified [13, 15, 18,
35].
6. Objective: Information is presented in objective
manner without political, cultural, religious, or
institutional biases [15, 18, 35].
7. Authority: The credibility or the level of user
confidence of websites’ information is clearly
identified by providing information about: the
organization’s physical address [13, 20, 22],
sponsor(s) of the site [12, 15, 18, 19, 26, 32, 35],
manager(s) of the site [12, 18, 19, 35],
specification of sites’ manager(s) [12, 18],
Content User-friendlyOrganization
Timely
Relevant
Multilanguage/Culture
Variety of Presentation
Accuracy
Objective
Index
Mapping
Consistency
Links
Logo
Usability
Reliability
Security /Privacy
Customization
Design
Attractive
Appropriateness
Color
Image/Sound/Video
Text
Website Quality
Authority
Interactive Features
Domain
DimensionsIndicators](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-5-320.jpg)
![identification of copyright [35], email to manager
of the website exists [13, 15, 19, 35], and
metadata elements exist [9, 12].
Table 1. Indicators and check elements of the content
quality dimension.
Indicators Checklist
Up-to-date information
How frequency the website is updated
Timely
When the website was updated
Organization’s objectives
Organization’s history
Customers (audience)
Products or services
Relevant
Photography of organization’s facilities
Use different languagesMultilanguage/
Culture Present to different cultured
Variety of
Presentation
Different forms (text, audio, video, …)
Precise information (no spelling, grammar
errors)
Accuracy
Sources of information is identified
Objective Objective presentation of information
Organization’s physical address
Sponsor (s) of the site
Manager (s) of the site
Specifications of site’s managers
Identification of copyright
Authority
Email to manager
4.2 Design Quality
This dimension concerns with the visual
characteristics of websites’ design that attract the
users and encourage them to stay longer time viewing
the website and reenter it. Most previous studies cover
this dimension for its importance. Companies put a
great effort to design their websites in an attractive
and innovative way since poor design can mean that
potential readers never see excellent material as they
may become bored, confused, and eventually abort
their attempt to view the information [39]. Yoo and
Jin [48] explained extensively twelve characteristics
that concentrate on website design dimension. Some
researchers consider this dimension as an important
dimension in their evaluating model. They called it
design, display, or presentation of information [4, 18,
19, 22, 27, 38-40, 43], while others described this
dimension as part of the usability dimension of their
criteria [5, 6, 24, 29].
The following indicators and check elements,
which are summarized in Table 2, are the most
important relating to the design quality dimension.
1. Attractive: The design of the website is
innovative [23], has an aesthetic effect by its
graphics and animation [2, 4-8, 13, 19, 22, 23, 36,
39, 43, 47]. It has an emotional appeal which
makes the user happy, pleasant, enjoyable, and
cheerful when visiting the website [4-7, 12, 23,
25, 28, 39, 49].
2. Appropriateness: The design of the website is
appropriate to the type of the website [5]. Images
used within the pages serve their functional
purposes [18, 35]. Images, colors, and text are
appropriately balanced on each page [19], and a
fewer number of screens in each page [22, 48].
3. Color: This concern with the effective use of
background and text colors when designing the
website [1, 6, 8, 19, 22, 24, 27, 39, 40, 43].
According to background color, light colors are
preferred to be used [47]. Concerning text color,
it shouldn’t exceed four colors within the same
page [48].
4. Image/Sound/Video: It concerns with the non text
elements which are used within the website [2, 6,
8, 18, 19, 22-24, 32, 35, 36, 39, 40, 43]. Few
number of image/sound/video should be used and
the size should be small since large size of
image/sound/video per page will slow
downloading the page which is not preferred by
users [38, 48]. Alternative text should be used for
all non-text elements [22, 29, 38].
5. Text: It concerns with the characteristics of text
used within websites’ pages [6, 8, 24, 40, 43].
There should be consistency in text; pages should
use one font size and one font style except for
titles [48]. Text font should be chosen among the
most readable ones [1, 22, 38] with relative size
[19, 38]. Pages shouldn’t use all capital letters
unless in titles or headings since they are hard to
be read and space wasting [38, 48]. Pages should
use white space or breathing space between page
elements to avoid crowded pages [27, 48].
Different or multiple headings such as titles, sub
titles, sub sub titles are preferred as appropriate
[48]. If pages use scrolling text, it shouldn’t hide
a large amount of information [48]. Pages should
show the text first then the image(s) to see text
while downloading image(s) [48].
Table 2. Indicators and check elements of the design
quality dimension.
Indicators Checklist
Innovative
Aesthetic effects
Attractive
Emotional appeal
Appropriate to the type of website
Image used within it serve functional
purposes
Balancing (images, colors, and text)
Appropriateness
Number of screens per page
Background colorColor
Text color
Number of image/sound/video
Size of image/sound/video
Image/Sound/
Video
Provide alternative text for all non
text elements
Consistency (type, style)
Readable
Relative size
Capital letters
Breathing space
Multiple headings
Scrolling text
Text
Sequential appearance of text then
images](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-6-320.jpg)
![4.3 Organization Quality
This dimension concerns with the logical grouping,
categorization, or structure of websites’ elements in
order to help the user to reach the required
information quickly, navigate easily within the
website, feel comfortable within its layout
consistency, and keep him/her informative that he/she
is still in the same website [1, 22, 32, 43, 46].
Heimlich and Wang [19] proposed structure themes
that extensively cover most elements of this
dimension. Most researchers referred to the elements
of organization dimension as part of usability
dimension in their models [2, 11, 36, 39, 40, 47].
While others referred to the elements of organization
dimension as part of other dimensions like
information [2], communication [20], content [38], or
navigation [6, 38, 43].
The following indicators and check elements,
which are summarized in Table 3, are the most
important relating to the organization quality
dimension.
1. Index: An index or links to all the website’s pages
is available from the main page, so that the user
will have an idea about all main categories of the
website [12, 19, 47].
2. Mapping: Adequate website map or navigation
bar/menu is available in each page to facilitate
navigating the website [2, 8, 12, 22, 26, 36, 38].
A user can know the current page that he/she is in
while browsing from the navigation title [8, 19,
22, 29].
3. Consistency: A general layout of each page is
consistent through the website [6, 11, 19, 22, 24,
47].
4. Links: Links work properly; it should take the
user where he/she is intended to go [13, 15, 18,
20, 22, 28, 38-49]. Assistant links should be
available in each page so that the user can get
back to the main page from every section of the
website; it can help the user to return to top of the
page within the long pages of the website and can
return to the original website when he/she follows
external link of any page [6, 12, 19]. Worthy
links that take user to other related websites are
available [2, 15, 22, 39], no dead links [20, 38],
and the link’s color is changed after the user has
visited it [19].
5. Logo: Organization logo is clear and noticeable in
every page of the website [26].
6. Domain: Organization has a meaningful name
related to the service it provides and according to
the net standards[17].
Table 3. Indicators and check elements of the
organization quality dimension.
Indicators Checklist
Index Index or links to all website’s pages
Adequate website map or navigation
bar/menu
Mapping
Current page
Consistency General layout
Working links
Assistant links (back to home, top, back to
original website)
Worthy links (to other related websites, no
dead links)
Links
Visiting pages
Logo Organization’s logo is clear and noticeable
Domain Meaningful domain name
4.4 User-friendly Quality
Nearly all previous studies included this dimension or
at least one of its indicators in their criteria model
because of its importance. It concerns with many
issues that help any user regardless of his/her
education or experience to find the needed
information within a reasonable time [24], the
capability of the website to maintain specific level of
performance when used [14], and interactivity or
connectivity which emphasize the existence of
interaction between user and website using different
tools.
The following indicators and check elements,
which are summarized in Table 4, are the most
important relating to the design user-friendly
dimension.
1. Usability: The website is easy to use, understand,
operate, find information, or navigate [1, 3-6, 8,
10-14,18,19,22-25,27-
29,33,36,38,40,41,43,47,49]. It is easy to find the
website using external websites [6, 18, 32, 41],
and it is clear to the user that new information is
added to the website.
2. Reliability: Website’s address is appropriate and
easy to remember [13, 15, 32, 44], short
download time [9, 10, 13, 22, 32, 33, 36, 39, 43,
46], multi browser support [1, 22, 29], and work
properly using different screen settings. Few ads
are in the website’s pages to avoid long time
downloading of website’s pages [2] and there is
away to measure its efficiency by counting the
number of visitors [6, 10]. Also, the website is
available 7 days/week, 24 hours/day [1, 3, 6, 10,
11, 13, 14, 33, 46].
3. Interactive features: The website has clear
instructions to use different parts/sections/forms
of it [4]. Help function and clear error messages
are available to help users [2, 15, 18, 27, 28, 38,
39, 41, 47]. FAQ is available that summarizes
frequently asked questions and their answers [2,
20, 26, 27, 33, 36, 44]. Effective internal search
tool to search the content of the website is
available [2, 3, 6, 7, 11, 13, 16, 18-20, 22, 23, 26,
28, 36, 48, 49]. Communication channel and
feedback exist between user and website through
email, chat rooms, online community, or
suggestion form [2, 3, 5-7, 10-14, 16, 18-20, 22,
23, 26, 27, 30, 32, 33, 36, 39, 44, 47]. Follow-up
service is offered and users can track their order
easily [28].
4. Security/Privacy: In order to gain users’ trust,
effective mechanisms are used to keep the](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-7-320.jpg)
![transactions secure [2-5, 10-12, 14, 15, 20, 22,
23, 25-28, 33, 36, 39, 49]. In order to gain users’
confidence, privacy of personal information is
needed so that information can’t be handled or
read by unauthorized users [2, 5, 6, 11, 15, 22,
23, 25-28, 33, 36].
5. Customization: The process of tailoring the
content of the website according to the needs and
performance of specific users [2-6, 10, 11, 13, 16,
30, 32, 39, 46].
Table 4. Indicators and check elements of the user-
friendly quality dimension.
Indicators Checklist
Ease to use, understand, operate, find, or
navigate
Easy to find using search engines
Usability
What’s new
Appropriate and easy to remember URL
Short download speed
Multi browser support
Work properly using different screen
settings
Fewer ads
Measuring efficiency
Reliability
Availability
Clear instructions
Help function
FAQ
Effective internal search tool
Feedback between user and website
(email, chat, online community,
suggested forms)
Review transactions
Interactive
Features
Tracking order
Secure transactionsSecurity
/Privacy Privacy
Customization Tailoring content to the needs of specific
users
5. Conclusions and Future Work
Recently, the Internet has shown a rapid growth in
term of commercial trade volume which led to a new
definition of almost all aspects of business. The
deployment of recent information and communication
technologies produced new era in business, finance,
and economics. The Internet created a new business
environment far different from anything that has come
before. The explosion of the web has determined the
need of measurement criteria to evaluate the aspects
related to the quality of web applications. Awareness
of quality issues has affected every industrial sector in
recent years, since an organization with a website that
is difficult to use and interact with, gives a poor image
on the Internet and weaken an organization’s position.
Therefore, it is important of an organization to assess
the quality of its e-commerce service, in order to
improve its services over time and benchmark against
competitors and best practices in any industry.
This paper reviewed the most recent evaluation
methods, which were used in evaluating the quality of
different websites, and proposes a comprehensive
framework for assessing the quality of any website
regardless of the type of service that it offers. The
dimensions of the framework along with their
indicators and checklist are summarized in the
Appendix below. Furthermore, our framework is
capable of reliable applications across a broad range
of websites regardless of the service they provide.
These dimensions with their indicators, after being
given certain weights, could be easily converted into a
questionnaire. The questionnaire could be applied to
different website domains such as business, education,
banking, commerce, government, and others. Results
from the analysis of the questionnaire will help in
evaluating these dimensions and their indicators and
make the needed update on them.
References
[1] Abanumy A., Al-Badi A., and Mayhew P. e-
Government website accessibility: in-depth evaluation
of Saudi Arabia and Oman, The Electronic Journal of
e-Government, v. 3, n. 3, p. 99-106, 2005.
[2] Achour H. and Bensedrine N. An evaluation of
internet banking and online brokerage in Tunisia, in
Proceedings of the 1st
International Conference on E-
Business and E-learning (EBEL), Amman, Jordan, p.
147-158, 2005.
[3] Awamleh R. and Fernandes C. Internet banking:
an empirical investigation into the extent of adoption
by banks and the determinants of customer
satisfaction in the United Arab Emirates, Journal of
Internet Banking and Commerce, v. 10, n. 1, 2005.
[4] Barnes S. and Vidgen R. Assessing the quality of
auction web sites, in Proceedings of the 34th
International Conference on System Sciences, 2001.
[5] Barnes S. and Vidgen R. An integrative approach
to the assessment of e-commerce quality, Journal of
Electronic Commerce Research, v. 3, n. 3, p. 114-127,
2002.
[6] Basu A. Context-driven assessment of commercial
web sites, in Proceedings of the 35th
Hawaii
International Conference on System Sciences, 2002.
[7] Cao M. and Zhang Q. Evaluating e-commerce web
site design: a customer’s perspective, in Proceedings
of Decision Sciences Institute 2002 Annual Meeting,
p. 1186-1191, 2002.
[8] Chanaron J. Evaluating e-learning: the case of
automative small-medium suppliers, in Proceedings of
the 1st
International Conference on e-Business and E-
learning (EBEL), Amman, Jordan, p. 13-25, 2005.
[9] Choudrie J., Ghinea G., and Weerakkody V.
Evaluating global e-government sites: a view using
web diagnostic tools," Electronic Journal of e-
Government, v. 2, n. 2, p. 105-114, 2004.
[10] Delone W. and Mclean E. The DeLone and
McLean of information systems success: a ten-year
update, Journal of Management Information Systems,
v. 19, n. 4, p. 9-30, 2003.
[11] Diniz E., Porto M. R., and Adachi T. Internet
banking in Brazil: evaluation of functionality,
reliability, and usability, The Electronic Journal of](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-8-320.jpg)
![Information Systems Evaluation, v. 8, n. 1, p. 41-50,
2005.
[12] Fitzpatrick R. Additional quality factors for the
World Wide Web, in Proceedings of the 2nd
World
Congress for Software Quality, Japan, 2000.
[13] Fogg B. J., Marshall J., Laraki O., Osipovich A.,
Varma C., Fang N., Paul J., Rangnekar A., Shon J.,
Swani P., and Treinen M. What makes web sites
credible? a report on a large quantitative study,
Computer Human Interaction, v. 3, n. 1, p. 61-68,
2001.
[14] Gledec G. Evaluating web site quality, in
Proceedings of the 7th
Internet Users Conference
(CUC2005), Croatia, 2005.
[15] Granath K. Evaluating web pages, available at:
http://www.lib.umt.edu/research/guide/int_evalweb.ht
m, 2006.
[16] Guru K. B., Shanmugam B., Alam N., and Perera
J. C. An evaluation of internet banking sites in Islamic
countries, Journal of Internet Banking and Commerce,
v. 6, n. 1, 2001.
[17] Hasan L. and Abuelrub E. Criteria for evaluating
quality of websites, in Proceedings of the 6th
IBMA
International Conference on Managing Information in
Digital Economy, Germany, 2006.
[18] Heimlich J. Evaluating the content of web sites,
Environmental Education and Training Partnership
Resource Library, Ohio State University Extension,
USA, 1999.
[19] Heimlich J. and Wang K. Evaluating the
structure of web sites, Environmental Education and
Training Partnership Resource Library, Ohio State
University Extension, USA, 1999.
[20] Hussin H., Suhaimi M., and Mustafa M. e-
commerce and ethical web design: applying the
BBBOnline guidelines on Malaysian web sites, The
International Arab Journal of Information
Technology, v. 2, n. 3, p. 218-226, 2005.
[21] International Trade Center UNCTAD/WTO and
Jordan Export Development & Commercial Centres
Corporation (JEDCO) Secrets of electronic
commerce: a guide for small and medium sized
exporters, Amman, Jordan, 2001.
[22] Kokkinaki I. A., Mylonas S., and Mina S. e-
government initiatives in Cyprus, e-Government
Workshop (eGOV05), Brunel University, UK, 2005.
[23] Krauss K. Testing an e-government website
quality questionnaire: a pilot study, in Proceedings of
the 5th
Annual Conference on World Wide Web
Applications (WWW2003), 2003.
[24] Lautenbach M. A., Schegget I. S., Schoute A. M.,
and Witteman C. L. Evaluating the usability of web
pages: a case study, available at:
http://www.phil.uu.nl/preprints/ckipreprints/PREPRI
NTS /preprint011.pdf, 2006.
[25] Lim K. Security and motivational factors of e-
shopping web site usage, in the Proceedings of
Decision Sciences Institute 2002 Annual Meeting, p.
611-616, 2002.
[26] Lin F., Huarng K., Chen Y., and Lin S. Quality
evaluation of web services, in the Proceedings of the
IEEE International Conference on e-Commerce
Technology for Dynamic e-Business, 2004.
[27] Lin O. and Joyce D. Critical success factors for
online auction web sites, in the Proceedings of the 17th
NACCQ, 2004.
[28] Liu C. and Arnett K. Exploring the factors
associated with web site success in the context of
electronic commerce, Information and Management,
v. 38, p. 23-33, 2000.
[29] Ma H. and Zaphiris P. The usability and content
accessibility of the e-government in the UK, in
Proceedings of Human Computer Interaction
International Conference, Greece, 2003.
[30] Madeja N. and Schoder D. Designed for success:
empirical evidence on features of corporate web
pages, in the Proceedings of the 36th
Hawaii
International Conference on System Sciences, 2003.
[31] Mendo A. The evolution of SMEs websites in the
UK, in the Proceedings of the 9th
European
Conference on Software Maintenance and
Reengineering, 2005.
[32] Mich L., Franch M., and Gaio L. Evaluating and
designing web site quality, Feature Article, IEEE
Multimedia, IEEE Computer Society, 2003.
[33] Molla A. and Licker S. P. e-commerce systems
success: an attempt to extend and respecify the
Delone and Maclean of IS success, Journal of
Electronic Commerce Research, v. 2, n. 4, p. 131-141,
2001.
[34] Moodle S., Morris M., and Kaplinsky R. School
of development studies: incorporating the centre for
social & development studies, University of Natal,
2007.
[35] Osborne C. and Rinalducci J. Evaluation of web
based resources within the art history discipline,
Technical Report, University of North Carolina, 2002.
[36] Paynter J. and Chung W. An evaluation of
internet banking in New Zealand, in Proceedings of
the 35th
Hawaii International Conference on System
Sciences, 2002.
[37] Pew Internet and American Life Project. Senior
Research Fellow Deborah Fellows, USA, 2006.
[38] Signore O. A comprehensive model for web sites
quality, in the Proceedings of the 7th
IEEE
International Symposium on Web Site Evolution
(WSE’05), 2005.
[39] Singh I. and Sook A. An evaluation of the
usability of South African University web sites, in
Proceedings of the 2002 CITTE Conference, Durban,
South Africa, 2002.
[40] Singh M. and Fisher J. Electronic commerce
issues: a discussion of two exploratory studies, in the
Proceedings of the Electronic 3rd
Annual Conference
on Electronic Commerce, Victoria University, New
Zealand, 1999.
[41] Srivihok A. An assessment tool for electronic
commerce: end user evaluation of web commerce](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-9-320.jpg)
![sites, Technical Report, Faculty of Science, Kasetsart
University, Bangkok, Thailand, 2000.
[42] Sui D. and Rejeski D. Environmental impacts of
the emerging digital economy: the e-for-
environmental e-commerce?, Environmental
Management, v. 29, n. 2, p. 155-163, 2002.
[43] Tan F. and Tung L. Exploring website evaluation
criteria using the repertory grid technique: a web
designers’ perspective, in the Proceedings of the 2nd
Annual Workshop on HCI Research in MIS, WA,
2003.
[44] Vijayan P. and Shanmugam B. Service quality
evaluation of internet banking in Malaysia, Journal of
Internet Banking and Commerce, v. 8, n. 1, 2003.
[45] Vlosky R. and Westbrook T. The state of forest
products industry e-business, World Wide Web
Network, Ltd, 2007.
[46] Webpartner, available at: www.webpartner.com,
2007.
[47] Wenham D. and Zaphiris P. User interface
evaluation methods for internet banking web sites: a
review, evaluation, and case study, in Jacko and
Stephanidis J. C. (Eds.), Human-Computer
Interaction: Theory and Practice, p. 721-725,
Lawrence Erlbaum, Mehwah, USA, 2003.
[48] Yoo S. and Jin J. Evaluation of the home page of
the top 100 university web sites, Academy of
Information and Management Sciences, v. 8, n. 2, p.
57-69, 2004.
[49] Zhang P. and Dran G. Expectations and ranking
of website quality features: results of two studies on
user perceptions, in the Proceedings of the 34th
Hawaii International Conference on System Sciences,
2001.
Appendix
Dimensions, indicators, and checklist of the proposed framework.
Dimensions Indicators Checklist
Up-to-date information
How frequency the website is updated
Timely
When the website was updated
Organization’s objectives
Organization’s history
Customers (audience)
Products or services
Relevant
Photography of organization’s facilities
Use different languagesMultilanguage/Culture
Present to different cultured
Variety of Presentation Different forms (text, audio, video, …)
Precise information (no spelling, grammar errors)Accuracy
Sources of information is identified
Objective Objective presentation of information
Organization’s physical address
Sponsor (s) of the site
Manager (s) of the site
Specifications of site’s managers
Identification of copyright
Content
Authority
Email to manager
Innovative
Aesthetic effects
Attractive
Emotional appeal
Appropriate to the type of website
Image used within it serve functional purposes
Balancing (images, colors, and text)
Appropriateness
Number of screens per page
Background colorColor
Text color
Number of image/sound/video
Size of image/sound/video
Image/Sound/Video
Provide alternative text for all non text elements
Consistency (type, style)
Readable
Relative size
Design
Text
Capital letters](https://image.slidesharecdn.com/assessingthequalityofwebsites-181217064955/85/Assessing-the-quality-of-websites-10-320.jpg)
Assessing the quality of websites
- 1. See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/228686530 Assessing the Quality of Web Sites Article · January 2011 DOI: 10.1016/j.aci.2009.03.001 CITATIONS 119 READS 1,860 2 authors: Some of the authors of this publication are also working on these related projects: Usability of e-learning systems: the case of Moodle View project Usability of Educational Websites View project Layla Hasan Zarqa University 32 PUBLICATIONS 359 CITATIONS SEE PROFILE Emad Abuelrub Ministry of Higher Education and Scientific Research 22 PUBLICATIONS 181 CITATIONS SEE PROFILE All content following this page was uploaded by Layla Hasan on 11 October 2015. The user has requested enhancement of the downloaded file.
- 2. Assessing the Quality of Web Sites Layla Hasan1 and Emad Abuelrub2 1 Department of Information Science, Loughborough University, UK 2 Department of Computer Science, Zarqa Private University, Jordan - Abstract: In the age of information technology and with the rapid growth of the World Wide Web, the spectacular growth trend in e-business, that has been experienced so far, is expected to continue. Since the introduction of the first commercial website in 1994, electronic commerce has spread across the globe as a marketing, sales, and communication phenomenon, changing the fact of some business sectors. Companies seeking to achieve significant benefits through e-business need to create an effective and usable web presence to ensure successful interaction and communication with their employees, partners, and customers. Web services are being widely deployed throughout business, education, government, and other organizations. The Internet creates a new business environment, far different from anything that has come before; because it lets a company conducts its entire set of business processes and practices online. As the dependency on web technology increases, the need to assess the factors associated with website success increases as well. This paper reviewed the most recent evaluation criteria methods, which were used in different e-business services. Furthermore, it proposes general criteria for evaluating the quality of any website regardless of the type of service that it offers. The dimensions of the criteria are content quality, design quality, organization quality, and user-friendly quality. These dimensions together with their comprehensive indicators and checklist can be used by web designers and developers to create quality websites to improve the electronic service and then the image of any organization on the Internet. Keywords: e-commerce, criteria, framework, website evaluation, website quality. (Received May 29, 2008 / Accepted October 29, 2008) 1. Introduction Since the introduction of the first electronic web service in the mid nineties of the last century, electronic services have spread across the globe in various shapes, changing the face of many business sectors. The new e-revolution is not only driving the global economics but also transforming societies into a knowledge-based economics all over the world. In the last few years, the Internet has shown a rapid growth in term of commercial trade volume, which led to a new definition of almost all aspects of business. The deployment of recent information and communication technologies enabled firms to provide higher quality services, lower prices for customers, and increase the profit margins for businesses. Also, the new technology created new trends in business and produced new era in business, finance, and economics [21, 42]. The Internet created a new business environment, far different from anything that has come before, enabling any company to conduct its entire set of business processes and practices online [45]. E- business is any business process performed via an Internet-based, computer-mediated network [34]. There are many categories of e-business; for example Business to Business (B2B), Business to Consumer (B2C), Consumer to Business (C2B), and Consumer to Consumer (C2C). E-business and e-commerce tend to be used interchangeably leading to policy incoherence. Recently, we have seen a proliferation of electronic websites with a tremendous amount of information either with high quality, or with low quality, as well as sites that are outright misleading [13, 18]. The number of websites grew from 17 million in the middle of 2000 to 65 million in the middle of 2005 [37]. The explosion of the web has determined the need of measurement criteria to evaluate the aspects related to the quality in use, such as usability and accessibility of a web application. The objective is to make a website useful, profitable, user linking, and accessible [38]. Awareness of quality issues has recently affected every industrial sector [32]. An organization with a website that is difficult to use and interact with, gives a poor image on the Internet and weaken an organization’s position. Therefore, it is important for any organization to have the ability to make an assessment of the quality of their e-commerce service, in order to improve their offerings over time and benchmark against competitors and best practices in any industry [5]. In the last decade, numerous studies have focused on the designs of websites for general information seeking purposes and for electronic commerce purposes [49]. The design and commercial development of websites are very critical to e- commerce success [5]. Numerous practitioner reports l.hasan2@lboro.ac.uk abuelrub@zpu.edu.jo
- 3. and reviews have been published seeking to identify the good and bad features of websites. Site reviews range from theoretical and scientific opinions to surveys of successful sites and features to existing successful e-businesses. Although, there has been a significant research on supporting electronic commerce, most of the existing empirical research focusing on success factors of websites is mainly exploratory in nature [28]. Up to our knowledge, there is no standard framework or benchmark defining website effectiveness [6]. This paper reviewed the most recent evaluation criteria methods which were used in different websites, and proposes a general comprehensive framework for evaluating the quality of any web service regardless of the type of service that it offers. The remaining of the paper is as follows. Section 2 gives a brief review of previous work. Section 3 discusses and analyzes the proposed framework. Section 4 defines the dimensions of the proposed framework and its indicators. Section 5 concludes the paper and suggests some future work. 2. Previous Work As the dependency on web services increases, the need to assess characteristics with website quality and success increases. Websites characteristics are important; they have been a constant concern of research in different domains and they were widely studied in the e-commerce literature [17]. Although there has been a significant research on supporting e- commerce, many existing empirical studies focusing on the quality of websites is mainly exploratory in nature. Most of the current studies are either dealing with a limited number of quality factors or directed toward a specific web service. Thus, while there should be a considerable number and variety of factors associated with web site success, little research exists about the combination of these factors and services. Recently, research and studies are accumulating including different models and/or frameworks to evaluate the quality and performance of websites. We categorized the previous studies that investigated the quality dimensions of websites according to the type of service that the website offers; for example business and commercial, educational, banking, governmental, and others. This section briefly reviews the previous studies according to the website service. Business and commercial websites were studied from different perspectives. Some researchers investigated website features or factors that are critical to e-business success, in which they called them critical success factors [10, 27, 28, 30, 33]. Other researchers address key issues, ideas and strategies to be considered in the management of online business from customer satisfaction perspective, and they assess whether a website has been built with a customer’s goals in mind [8, 19, 41, 46, 49]. Another group of researchers investigated the perspective of web designers in order to elicit factors that they consider important when designing or developing effective websites [8, 43]. Other researchers developed generic tools or measurement frameworks for the assessment of website quality [4, 5, 12, 14, 26, 32, 38]. Some researchers concentrated on some important features; they either proposed a framework to measure the important features of the website or used previous models to find out to which extent e-business websites incorporate these important features. Others [6, 25, 40] concentrated in their studies on website usability, Heimlich and Wang [19] proposed key issues of website’s structure, while Cao and Zhang [7] examined factors that affect e- commerce website design. Heimlich [18] discussed content evaluation of websites, Hussin et al. [20] studied the extent in which companies incorporate ethical and trustworthy elements on their websites, while Fogg et al. [13] investigated how different elements of websites affect people’s perception of credibility. Educational websites were also studied from many different perspectives. Zhang and Dran [49] developed a theoretical framework for evaluating website quality from a user satisfaction perspective. Others concentrated on some specific features of websites. For example Lautenbach et al. [24] developed a framework to measure usability of websites, while Yoo and Jin [48] investigated and evaluated the design of university websites. Other researchers, while assessing the university websites, took in consideration other features. Osborne and Rinalducci [35] designed a criteria to evaluate web resources for utilization within the context of scholarly research within the discipline of the art history. Singh and Sook [39] attempted to find solutions to user problems and involved evaluating South African university websites on certain factors. Banking websites were studied from many different perspectives using different models. Diniz et al. [11] proposed a model to evaluate and build digital business environment from the user’s point of view. While other researchers proposed a specific framework to evaluate the Internet banking websites and the service quality of Internet banking [2, 44, 47]. Other researchers adopted through their studies a number of previous models to evaluate Internet banking websites. For example, Awamleh and Fernandes [3] used Diniz Model to evaluate websites of foreign and local banks in the United Arab Emirates. Guru et al. [16] evaluated the web presence of banks in Islamic countries based on a Diniz Model too. Paynter and Chung [36] examined how New Zealand banks enhanced their retail banking services through the Internet using Herey’s Model for website evaluation. Governmental websites were studied from different perspectives. Zhang and Dran [49] developed a theoretical framework for evaluating website quality from a user satisfaction perspective
- 4. through theoretical and empirical investigations. While Krauss [23] identified seventeen comprehensive quality dimensions that can be used to rate website quality features that are important to e- government websites. Others researchers concentrated on other features. For example Kokkinaki et al. [22] presented a framework for evaluating existing e- government initiatives in Cyprus. It includes content characteristics, design characteristics, and common features of e-government websites. Choudrie et al. [9] described the issues related to the accessibility, quality, and privacy of government web using a common set of performance metrics and web diagnostic tools which are WebXact, Ntmechanic, and Vizcheck. Ma and Zaphiris [29] studied the usability and content accessibility of UK e-government websites and investigated whether they are ranked high in terms of accessibility. The usability and accessibility of fifty selected UK e-government websites were measured using two automatic evaluation tools, Bobby and LIFT. Abanumy et al. [1] investigated the issue that makes a website accessible and explored the importance placed on web accessibility with respect to e-government websites. Other researchers addressed other kinds of web services from different perspectives. Lin and Joyce [27] studied different e-commerce models of online auction websites. Six critical success factors for a successful online auction website were identified. They include design and content, consumer education, security, customer support, online community, and market positioning. Barnes and Vidgen [4] deployed WEBQUAL in the domain of Internet auctions and identified three quality dimensions; information, interaction, and site design. Lim [25] evaluated the impact of four relevant factors to e-shopping; usefulness, ease of use, enjoyment, and security. Results showed that perceived ease of use and usefulness of the e-shopping website have a significant direct impact on the success of the site. 3. Discussion and Analysis The objective of this research is to develop a theoretical, comprehensive, and measurable framework for assessing the quality of websites in order to provide straight forward criteria to encourage improvements of website design and its implementation. Furthermore, we aim to develop a framework that is capable of reliable applications across a broad range of websites regardless of the service they provide. A multi-phase approach was adopted that included a wide range of literature review, review of leading sites, identification of success factors from research and industry literature, comparison of factors with published industry scoring studies, and using our own experience in the field. Our process overlaid industry and academic research to identify quality factors in order to meet the objectives of this research. After deep and comprehensive review of different evaluation methods and their elements that were used in different services over the Internet, we propose 4- dimensions criteria which is comprehensive and includes all previous dimensions and elements, in order to be used as general criteria to evaluate all kinds of websites. The dimensions of the proposed criteria are content quality, design quality, organization quality, and user-friendly quality. In order to investigate how our proposed criteria were used in previous studies, we re-arranged each element of each dimension of the previous work to be under one of the four new dimensions. The result of re- arranging the dimensions of previous work into the proposed 4-dimensions criteria is shown in Figure 1. Content Quality; 37; 29% Design Quality; 30; 23% Organization Quality; 22; 17% Usr-Friendly Quality; 40; 31% Content Quality Design Quality Organization Quality Usr-Friendly Quality Figure 1. Common dimensions after re-arrangement. We summarized the common quality dimensions according to the service that a website offers. Common dimensions that were used in evaluating e- business and e-commerce websites were concentrating on currency, accuracy, comprehensive and value added content, ease of use, reliability, availability of the needed information, speed of downloading, customization, effective internal search, different types of service and support to customers, security and privacy in all types of transactions, logical grouping of website elements, and attractive design that will attract the user and encourages him/her to spend more time in the website [4-7, 10, 12-14, 18-20, 25-28, 30, 32, 33, 38, 40, 41, 43, 46, 49]. Common dimensions that were used to assess the quality of educational websites were concentrated on currency, accuracy and comprehensibility of information, ease of use, clear layout of websites, and an attractive design [24, 35, 39, 48, 49]. Common dimensions that were used to assess the quality of banking websites focused on ease of use, customization, internal search function, security of transactions, aesthetic effects, and useful interaction between users and websites in order to get instant feedback from website to user [2, 3, 11, 16, 36, 44, 47, 49]. Common dimensions that were used to assess the quality of governmental websites are quick response time, up-to-date, accurate information, effective search tool, easy to understand, and secure transactions [1, 9, 22, 23, 29, 49]. Common
- 5. dimensions that were used to assess the quality of auctions and e-shopping websites are design, content, security, support, and ease of use [4, 25, 27]. 4. The Proposed Framework The proposed framework attempts to integrate knowledge and experience from disparate sources, a range of reference disciplines and empirical practices. The objective is to identify measurable features and indicators that currently comprise a successful web site. A set of features is developed that comprise a current representation of a perfect website. The proposed framework can be used to compare between the quality of websites, to identify a path for improvement of a website, and to provide a guideline for designers and developers when creating new websites. After we reviewed each evaluation criteria, we added its indicators to the suitable place of the proposed 4-dimension criteria, besides adding some indicators in which we see them important from our own experience. Our criteria include all main indicators of the previous studies of evaluating the quality of websites. Figure 2 summarizes the hierarchy of the proposed framework. Figure 2. Hierarchy of the proposed framework. 4.1 Content Quality It is generally agreed that content quality is an important dimension, which deals with the characteristics of websites’ information. Singh and Sook [39] called this dimension the king dimension of any website, since it is the major source of value to customers. This dimension has been addresses by a variety of researchers in different ways. Some researchers studied the content of websites without taking into consideration other dimensions [15, 18], while others considered content quality or information quality as one of the basic dimensions of their evaluating models [2, 4-6, 10, 22, 27, 28, 32, 33, 36, 38-40, 43]. The following indicators and check elements, which are summarized in Table 1, are the most important relating to the content quality dimension. 1. Timely: The currency of websites’ information and how much it is up-to-date, how frequently the website is updated, and is it clear when the site was updated [4, 5, 7, 9, 12, 13, 15, 18, 19, 22, 24, 28, 30, 31, 33, 35, 38, 41, 49]. 2. Relevant: The extent to which websites’ information is comprehensive, complete and provides the right level of details [5, 10, 12, 14, 15, 18, 32, 33, 35, 43, 49]. The extent to which it is informative, meaning, value added according to its audience [22, 46], and fits to users’ need [7, 22, 23, 32, 35, 39]. So, websites include information about the organization’s objectives [3, 19, 22, 36], organization’s history [3, 6], customers or audience [15, 18, 36], products or services [6, 28, 36], and photographs of organization’s facilities to reduce customer fears of dealing with the website [6]. 3. Multilanguage/Culture: The websites’ information is available in different languages [1, 12, 13, 22, 23, 26, 44], suitable to different cultures [12, 23], and meets the needs of all customers regardless of their county. 4. Variety of presentation: Information is presented in different forms (text (.doc, .pdf, … ), video, audio, … ), so that the user can download the form that suits him/her [22, 39, 44]. 5. Accuracy: Information is precise, there is no spelling errors or grammar errors [4, 5, 7, 12, 14, 15, 18, 22, 23, 28, 32, 33, 39, 41, 49] and the sources of information is identified [13, 15, 18, 35]. 6. Objective: Information is presented in objective manner without political, cultural, religious, or institutional biases [15, 18, 35]. 7. Authority: The credibility or the level of user confidence of websites’ information is clearly identified by providing information about: the organization’s physical address [13, 20, 22], sponsor(s) of the site [12, 15, 18, 19, 26, 32, 35], manager(s) of the site [12, 18, 19, 35], specification of sites’ manager(s) [12, 18], Content User-friendlyOrganization Timely Relevant Multilanguage/Culture Variety of Presentation Accuracy Objective Index Mapping Consistency Links Logo Usability Reliability Security /Privacy Customization Design Attractive Appropriateness Color Image/Sound/Video Text Website Quality Authority Interactive Features Domain DimensionsIndicators
- 6. identification of copyright [35], email to manager of the website exists [13, 15, 19, 35], and metadata elements exist [9, 12]. Table 1. Indicators and check elements of the content quality dimension. Indicators Checklist Up-to-date information How frequency the website is updated Timely When the website was updated Organization’s objectives Organization’s history Customers (audience) Products or services Relevant Photography of organization’s facilities Use different languagesMultilanguage/ Culture Present to different cultured Variety of Presentation Different forms (text, audio, video, …) Precise information (no spelling, grammar errors) Accuracy Sources of information is identified Objective Objective presentation of information Organization’s physical address Sponsor (s) of the site Manager (s) of the site Specifications of site’s managers Identification of copyright Authority Email to manager 4.2 Design Quality This dimension concerns with the visual characteristics of websites’ design that attract the users and encourage them to stay longer time viewing the website and reenter it. Most previous studies cover this dimension for its importance. Companies put a great effort to design their websites in an attractive and innovative way since poor design can mean that potential readers never see excellent material as they may become bored, confused, and eventually abort their attempt to view the information [39]. Yoo and Jin [48] explained extensively twelve characteristics that concentrate on website design dimension. Some researchers consider this dimension as an important dimension in their evaluating model. They called it design, display, or presentation of information [4, 18, 19, 22, 27, 38-40, 43], while others described this dimension as part of the usability dimension of their criteria [5, 6, 24, 29]. The following indicators and check elements, which are summarized in Table 2, are the most important relating to the design quality dimension. 1. Attractive: The design of the website is innovative [23], has an aesthetic effect by its graphics and animation [2, 4-8, 13, 19, 22, 23, 36, 39, 43, 47]. It has an emotional appeal which makes the user happy, pleasant, enjoyable, and cheerful when visiting the website [4-7, 12, 23, 25, 28, 39, 49]. 2. Appropriateness: The design of the website is appropriate to the type of the website [5]. Images used within the pages serve their functional purposes [18, 35]. Images, colors, and text are appropriately balanced on each page [19], and a fewer number of screens in each page [22, 48]. 3. Color: This concern with the effective use of background and text colors when designing the website [1, 6, 8, 19, 22, 24, 27, 39, 40, 43]. According to background color, light colors are preferred to be used [47]. Concerning text color, it shouldn’t exceed four colors within the same page [48]. 4. Image/Sound/Video: It concerns with the non text elements which are used within the website [2, 6, 8, 18, 19, 22-24, 32, 35, 36, 39, 40, 43]. Few number of image/sound/video should be used and the size should be small since large size of image/sound/video per page will slow downloading the page which is not preferred by users [38, 48]. Alternative text should be used for all non-text elements [22, 29, 38]. 5. Text: It concerns with the characteristics of text used within websites’ pages [6, 8, 24, 40, 43]. There should be consistency in text; pages should use one font size and one font style except for titles [48]. Text font should be chosen among the most readable ones [1, 22, 38] with relative size [19, 38]. Pages shouldn’t use all capital letters unless in titles or headings since they are hard to be read and space wasting [38, 48]. Pages should use white space or breathing space between page elements to avoid crowded pages [27, 48]. Different or multiple headings such as titles, sub titles, sub sub titles are preferred as appropriate [48]. If pages use scrolling text, it shouldn’t hide a large amount of information [48]. Pages should show the text first then the image(s) to see text while downloading image(s) [48]. Table 2. Indicators and check elements of the design quality dimension. Indicators Checklist Innovative Aesthetic effects Attractive Emotional appeal Appropriate to the type of website Image used within it serve functional purposes Balancing (images, colors, and text) Appropriateness Number of screens per page Background colorColor Text color Number of image/sound/video Size of image/sound/video Image/Sound/ Video Provide alternative text for all non text elements Consistency (type, style) Readable Relative size Capital letters Breathing space Multiple headings Scrolling text Text Sequential appearance of text then images
- 7. 4.3 Organization Quality This dimension concerns with the logical grouping, categorization, or structure of websites’ elements in order to help the user to reach the required information quickly, navigate easily within the website, feel comfortable within its layout consistency, and keep him/her informative that he/she is still in the same website [1, 22, 32, 43, 46]. Heimlich and Wang [19] proposed structure themes that extensively cover most elements of this dimension. Most researchers referred to the elements of organization dimension as part of usability dimension in their models [2, 11, 36, 39, 40, 47]. While others referred to the elements of organization dimension as part of other dimensions like information [2], communication [20], content [38], or navigation [6, 38, 43]. The following indicators and check elements, which are summarized in Table 3, are the most important relating to the organization quality dimension. 1. Index: An index or links to all the website’s pages is available from the main page, so that the user will have an idea about all main categories of the website [12, 19, 47]. 2. Mapping: Adequate website map or navigation bar/menu is available in each page to facilitate navigating the website [2, 8, 12, 22, 26, 36, 38]. A user can know the current page that he/she is in while browsing from the navigation title [8, 19, 22, 29]. 3. Consistency: A general layout of each page is consistent through the website [6, 11, 19, 22, 24, 47]. 4. Links: Links work properly; it should take the user where he/she is intended to go [13, 15, 18, 20, 22, 28, 38-49]. Assistant links should be available in each page so that the user can get back to the main page from every section of the website; it can help the user to return to top of the page within the long pages of the website and can return to the original website when he/she follows external link of any page [6, 12, 19]. Worthy links that take user to other related websites are available [2, 15, 22, 39], no dead links [20, 38], and the link’s color is changed after the user has visited it [19]. 5. Logo: Organization logo is clear and noticeable in every page of the website [26]. 6. Domain: Organization has a meaningful name related to the service it provides and according to the net standards[17]. Table 3. Indicators and check elements of the organization quality dimension. Indicators Checklist Index Index or links to all website’s pages Adequate website map or navigation bar/menu Mapping Current page Consistency General layout Working links Assistant links (back to home, top, back to original website) Worthy links (to other related websites, no dead links) Links Visiting pages Logo Organization’s logo is clear and noticeable Domain Meaningful domain name 4.4 User-friendly Quality Nearly all previous studies included this dimension or at least one of its indicators in their criteria model because of its importance. It concerns with many issues that help any user regardless of his/her education or experience to find the needed information within a reasonable time [24], the capability of the website to maintain specific level of performance when used [14], and interactivity or connectivity which emphasize the existence of interaction between user and website using different tools. The following indicators and check elements, which are summarized in Table 4, are the most important relating to the design user-friendly dimension. 1. Usability: The website is easy to use, understand, operate, find information, or navigate [1, 3-6, 8, 10-14,18,19,22-25,27- 29,33,36,38,40,41,43,47,49]. It is easy to find the website using external websites [6, 18, 32, 41], and it is clear to the user that new information is added to the website. 2. Reliability: Website’s address is appropriate and easy to remember [13, 15, 32, 44], short download time [9, 10, 13, 22, 32, 33, 36, 39, 43, 46], multi browser support [1, 22, 29], and work properly using different screen settings. Few ads are in the website’s pages to avoid long time downloading of website’s pages [2] and there is away to measure its efficiency by counting the number of visitors [6, 10]. Also, the website is available 7 days/week, 24 hours/day [1, 3, 6, 10, 11, 13, 14, 33, 46]. 3. Interactive features: The website has clear instructions to use different parts/sections/forms of it [4]. Help function and clear error messages are available to help users [2, 15, 18, 27, 28, 38, 39, 41, 47]. FAQ is available that summarizes frequently asked questions and their answers [2, 20, 26, 27, 33, 36, 44]. Effective internal search tool to search the content of the website is available [2, 3, 6, 7, 11, 13, 16, 18-20, 22, 23, 26, 28, 36, 48, 49]. Communication channel and feedback exist between user and website through email, chat rooms, online community, or suggestion form [2, 3, 5-7, 10-14, 16, 18-20, 22, 23, 26, 27, 30, 32, 33, 36, 39, 44, 47]. Follow-up service is offered and users can track their order easily [28]. 4. Security/Privacy: In order to gain users’ trust, effective mechanisms are used to keep the
- 8. transactions secure [2-5, 10-12, 14, 15, 20, 22, 23, 25-28, 33, 36, 39, 49]. In order to gain users’ confidence, privacy of personal information is needed so that information can’t be handled or read by unauthorized users [2, 5, 6, 11, 15, 22, 23, 25-28, 33, 36]. 5. Customization: The process of tailoring the content of the website according to the needs and performance of specific users [2-6, 10, 11, 13, 16, 30, 32, 39, 46]. Table 4. Indicators and check elements of the user- friendly quality dimension. Indicators Checklist Ease to use, understand, operate, find, or navigate Easy to find using search engines Usability What’s new Appropriate and easy to remember URL Short download speed Multi browser support Work properly using different screen settings Fewer ads Measuring efficiency Reliability Availability Clear instructions Help function FAQ Effective internal search tool Feedback between user and website (email, chat, online community, suggested forms) Review transactions Interactive Features Tracking order Secure transactionsSecurity /Privacy Privacy Customization Tailoring content to the needs of specific users 5. Conclusions and Future Work Recently, the Internet has shown a rapid growth in term of commercial trade volume which led to a new definition of almost all aspects of business. The deployment of recent information and communication technologies produced new era in business, finance, and economics. The Internet created a new business environment far different from anything that has come before. The explosion of the web has determined the need of measurement criteria to evaluate the aspects related to the quality of web applications. Awareness of quality issues has affected every industrial sector in recent years, since an organization with a website that is difficult to use and interact with, gives a poor image on the Internet and weaken an organization’s position. Therefore, it is important of an organization to assess the quality of its e-commerce service, in order to improve its services over time and benchmark against competitors and best practices in any industry. This paper reviewed the most recent evaluation methods, which were used in evaluating the quality of different websites, and proposes a comprehensive framework for assessing the quality of any website regardless of the type of service that it offers. The dimensions of the framework along with their indicators and checklist are summarized in the Appendix below. Furthermore, our framework is capable of reliable applications across a broad range of websites regardless of the service they provide. These dimensions with their indicators, after being given certain weights, could be easily converted into a questionnaire. The questionnaire could be applied to different website domains such as business, education, banking, commerce, government, and others. Results from the analysis of the questionnaire will help in evaluating these dimensions and their indicators and make the needed update on them. References [1] Abanumy A., Al-Badi A., and Mayhew P. e- Government website accessibility: in-depth evaluation of Saudi Arabia and Oman, The Electronic Journal of e-Government, v. 3, n. 3, p. 99-106, 2005. [2] Achour H. and Bensedrine N. An evaluation of internet banking and online brokerage in Tunisia, in Proceedings of the 1st International Conference on E- Business and E-learning (EBEL), Amman, Jordan, p. 147-158, 2005. [3] Awamleh R. and Fernandes C. Internet banking: an empirical investigation into the extent of adoption by banks and the determinants of customer satisfaction in the United Arab Emirates, Journal of Internet Banking and Commerce, v. 10, n. 1, 2005. [4] Barnes S. and Vidgen R. Assessing the quality of auction web sites, in Proceedings of the 34th International Conference on System Sciences, 2001. [5] Barnes S. and Vidgen R. An integrative approach to the assessment of e-commerce quality, Journal of Electronic Commerce Research, v. 3, n. 3, p. 114-127, 2002. [6] Basu A. Context-driven assessment of commercial web sites, in Proceedings of the 35th Hawaii International Conference on System Sciences, 2002. [7] Cao M. and Zhang Q. Evaluating e-commerce web site design: a customer’s perspective, in Proceedings of Decision Sciences Institute 2002 Annual Meeting, p. 1186-1191, 2002. [8] Chanaron J. Evaluating e-learning: the case of automative small-medium suppliers, in Proceedings of the 1st International Conference on e-Business and E- learning (EBEL), Amman, Jordan, p. 13-25, 2005. [9] Choudrie J., Ghinea G., and Weerakkody V. Evaluating global e-government sites: a view using web diagnostic tools," Electronic Journal of e- Government, v. 2, n. 2, p. 105-114, 2004. [10] Delone W. and Mclean E. The DeLone and McLean of information systems success: a ten-year update, Journal of Management Information Systems, v. 19, n. 4, p. 9-30, 2003. [11] Diniz E., Porto M. R., and Adachi T. Internet banking in Brazil: evaluation of functionality, reliability, and usability, The Electronic Journal of
- 9. Information Systems Evaluation, v. 8, n. 1, p. 41-50, 2005. [12] Fitzpatrick R. Additional quality factors for the World Wide Web, in Proceedings of the 2nd World Congress for Software Quality, Japan, 2000. [13] Fogg B. J., Marshall J., Laraki O., Osipovich A., Varma C., Fang N., Paul J., Rangnekar A., Shon J., Swani P., and Treinen M. What makes web sites credible? a report on a large quantitative study, Computer Human Interaction, v. 3, n. 1, p. 61-68, 2001. [14] Gledec G. Evaluating web site quality, in Proceedings of the 7th Internet Users Conference (CUC2005), Croatia, 2005. [15] Granath K. Evaluating web pages, available at: http://www.lib.umt.edu/research/guide/int_evalweb.ht m, 2006. [16] Guru K. B., Shanmugam B., Alam N., and Perera J. C. An evaluation of internet banking sites in Islamic countries, Journal of Internet Banking and Commerce, v. 6, n. 1, 2001. [17] Hasan L. and Abuelrub E. Criteria for evaluating quality of websites, in Proceedings of the 6th IBMA International Conference on Managing Information in Digital Economy, Germany, 2006. [18] Heimlich J. Evaluating the content of web sites, Environmental Education and Training Partnership Resource Library, Ohio State University Extension, USA, 1999. [19] Heimlich J. and Wang K. Evaluating the structure of web sites, Environmental Education and Training Partnership Resource Library, Ohio State University Extension, USA, 1999. [20] Hussin H., Suhaimi M., and Mustafa M. e- commerce and ethical web design: applying the BBBOnline guidelines on Malaysian web sites, The International Arab Journal of Information Technology, v. 2, n. 3, p. 218-226, 2005. [21] International Trade Center UNCTAD/WTO and Jordan Export Development & Commercial Centres Corporation (JEDCO) Secrets of electronic commerce: a guide for small and medium sized exporters, Amman, Jordan, 2001. [22] Kokkinaki I. A., Mylonas S., and Mina S. e- government initiatives in Cyprus, e-Government Workshop (eGOV05), Brunel University, UK, 2005. [23] Krauss K. Testing an e-government website quality questionnaire: a pilot study, in Proceedings of the 5th Annual Conference on World Wide Web Applications (WWW2003), 2003. [24] Lautenbach M. A., Schegget I. S., Schoute A. M., and Witteman C. L. Evaluating the usability of web pages: a case study, available at: http://www.phil.uu.nl/preprints/ckipreprints/PREPRI NTS /preprint011.pdf, 2006. [25] Lim K. Security and motivational factors of e- shopping web site usage, in the Proceedings of Decision Sciences Institute 2002 Annual Meeting, p. 611-616, 2002. [26] Lin F., Huarng K., Chen Y., and Lin S. Quality evaluation of web services, in the Proceedings of the IEEE International Conference on e-Commerce Technology for Dynamic e-Business, 2004. [27] Lin O. and Joyce D. Critical success factors for online auction web sites, in the Proceedings of the 17th NACCQ, 2004. [28] Liu C. and Arnett K. Exploring the factors associated with web site success in the context of electronic commerce, Information and Management, v. 38, p. 23-33, 2000. [29] Ma H. and Zaphiris P. The usability and content accessibility of the e-government in the UK, in Proceedings of Human Computer Interaction International Conference, Greece, 2003. [30] Madeja N. and Schoder D. Designed for success: empirical evidence on features of corporate web pages, in the Proceedings of the 36th Hawaii International Conference on System Sciences, 2003. [31] Mendo A. The evolution of SMEs websites in the UK, in the Proceedings of the 9th European Conference on Software Maintenance and Reengineering, 2005. [32] Mich L., Franch M., and Gaio L. Evaluating and designing web site quality, Feature Article, IEEE Multimedia, IEEE Computer Society, 2003. [33] Molla A. and Licker S. P. e-commerce systems success: an attempt to extend and respecify the Delone and Maclean of IS success, Journal of Electronic Commerce Research, v. 2, n. 4, p. 131-141, 2001. [34] Moodle S., Morris M., and Kaplinsky R. School of development studies: incorporating the centre for social & development studies, University of Natal, 2007. [35] Osborne C. and Rinalducci J. Evaluation of web based resources within the art history discipline, Technical Report, University of North Carolina, 2002. [36] Paynter J. and Chung W. An evaluation of internet banking in New Zealand, in Proceedings of the 35th Hawaii International Conference on System Sciences, 2002. [37] Pew Internet and American Life Project. Senior Research Fellow Deborah Fellows, USA, 2006. [38] Signore O. A comprehensive model for web sites quality, in the Proceedings of the 7th IEEE International Symposium on Web Site Evolution (WSE’05), 2005. [39] Singh I. and Sook A. An evaluation of the usability of South African University web sites, in Proceedings of the 2002 CITTE Conference, Durban, South Africa, 2002. [40] Singh M. and Fisher J. Electronic commerce issues: a discussion of two exploratory studies, in the Proceedings of the Electronic 3rd Annual Conference on Electronic Commerce, Victoria University, New Zealand, 1999. [41] Srivihok A. An assessment tool for electronic commerce: end user evaluation of web commerce
- 10. sites, Technical Report, Faculty of Science, Kasetsart University, Bangkok, Thailand, 2000. [42] Sui D. and Rejeski D. Environmental impacts of the emerging digital economy: the e-for- environmental e-commerce?, Environmental Management, v. 29, n. 2, p. 155-163, 2002. [43] Tan F. and Tung L. Exploring website evaluation criteria using the repertory grid technique: a web designers’ perspective, in the Proceedings of the 2nd Annual Workshop on HCI Research in MIS, WA, 2003. [44] Vijayan P. and Shanmugam B. Service quality evaluation of internet banking in Malaysia, Journal of Internet Banking and Commerce, v. 8, n. 1, 2003. [45] Vlosky R. and Westbrook T. The state of forest products industry e-business, World Wide Web Network, Ltd, 2007. [46] Webpartner, available at: www.webpartner.com, 2007. [47] Wenham D. and Zaphiris P. User interface evaluation methods for internet banking web sites: a review, evaluation, and case study, in Jacko and Stephanidis J. C. (Eds.), Human-Computer Interaction: Theory and Practice, p. 721-725, Lawrence Erlbaum, Mehwah, USA, 2003. [48] Yoo S. and Jin J. Evaluation of the home page of the top 100 university web sites, Academy of Information and Management Sciences, v. 8, n. 2, p. 57-69, 2004. [49] Zhang P. and Dran G. Expectations and ranking of website quality features: results of two studies on user perceptions, in the Proceedings of the 34th Hawaii International Conference on System Sciences, 2001. Appendix Dimensions, indicators, and checklist of the proposed framework. Dimensions Indicators Checklist Up-to-date information How frequency the website is updated Timely When the website was updated Organization’s objectives Organization’s history Customers (audience) Products or services Relevant Photography of organization’s facilities Use different languagesMultilanguage/Culture Present to different cultured Variety of Presentation Different forms (text, audio, video, …) Precise information (no spelling, grammar errors)Accuracy Sources of information is identified Objective Objective presentation of information Organization’s physical address Sponsor (s) of the site Manager (s) of the site Specifications of site’s managers Identification of copyright Content Authority Email to manager Innovative Aesthetic effects Attractive Emotional appeal Appropriate to the type of website Image used within it serve functional purposes Balancing (images, colors, and text) Appropriateness Number of screens per page Background colorColor Text color Number of image/sound/video Size of image/sound/video Image/Sound/Video Provide alternative text for all non text elements Consistency (type, style) Readable Relative size Design Text Capital letters
- 11. Breathing space Multiple headings Scrolling text Sequential appearance of text then images Index Index or links to all website’s pages Adequate website map or navigation bar/menuMapping Current page Consistency General layout Working links Assistant links (back to home, top, back to original website) Worthy links (to other related websites, no dead links) Links Visiting pages Organization Logo Organization’s logo is clear and noticeable Domain Meaningful domain name Ease to use, understand, operate, find, or navigate Easy to find using search engines Usability What’s new Appropriate and easy to remember URL Short download speed Multi browser support Work properly using different screen settings Fewer ads Measuring efficiency Reliability Availability Clear instructions Help function FAQ Effective internal search tool Feedback between user and website (email, chat, online community, suggested forms) Review transactions Interactive Features Tracking order Secure transactions User-friendly Security /Privacy Privacy View publication statsView publication stats