Valoris-Thesis

THESIS
A MIXED-METHODS ANALYSIS OF BEST PRACTICES FOR
LAND-GRANT UNIVERSITY MOBILE APPLICATIONS
FROM A USER EXPERIENCE DESIGN PERSPECTIVE
Submitted by
Michelle Valoris
Department of Journalism and Media Communication
In partial fulfillment of the requirements
For the Degree of Master of Science
Colorado State University
Fort Collins, Colorado
Summer 2015
Master’s Committee:
Advisor: Kirk Hallahan
Jangyul Kim
David McKelfresh

Copyright by Michelle Valoris 2015
All Rights Reserved

ii
ABSTRACT
A MIXED-METHODS ANALYSIS OF BEST PRACTICES FOR
LAND-GRANT UNIVERSITY MOBILE APPLICATIONS
FROM A USER EXPERIENCE DESIGN PERSPECTIVE
A content analysis of 30 mobile applications for the largest land-grant universities in the
United States was conducted to examine the use of recognized best practices for mobile
application development. A total of 49 variables were identified as best practices across five
hierarchical categories of user experience (UX) design based on a model proposed by Garrett
(2011). These included: visual design, user interface and navigation design, interaction design
and information architecture, content and services offered, and functional and technical
specifications. Based on quantitative scores for each of the variables, the best overall apps were
identified and reviewed using qualitative description analysis. The best overall app was created
by the University of New Hampshire; other top apps examined in the study were developed by
the University of Arizona, Ohio State University, University of Florida, Washington State
University and Michigan State University.
The findings revealed that it was very difficult for a mobile app to be exemplary in all
five of the UX categories and thus comply with best practices. Analysis of the top six ranked
applications portrayed how the UX categories and best practices interacted with and relied on
one another to create a successful user experience. The study provided guidelines for
universities to build more effective mobile apps and a foundation for further research into

iii
mobile application best practice measures, mobile app strategies, and the user experience of
mobile applications.

iv
ACKNOWLEDGEMENTS
Thanks to my family and friends who have supported me on this long journey through
graduate school and the ups and downs of life these past three years.

v
DEDICATION
This thesis is dedicated to my father who always believed in me.
While you he is not here to see this day finally come, I know that he is proud!

vi
TABLE OF CONTENTS
CHAPTER 1 INTRODUCTION ............................................................................................................................................. 1
Statement of the problem ................................................................................................................................................ 1
Rationale for the Study ..................................................................................................................................................... 1
Mobile Applications ........................................................................................................................................................... 2
Universities and Mobile: The why and how of going mobile ........................................................................... 4
CHAPTER 2 LITERATURE REVIEW ................................................................................................................................. 8
Best Practices Methodology ........................................................................................................................................... 8
User Experience (UX) ..................................................................................................................................................... 11
CHAPTER 3 RESEARCH QUESTIONS AND METHODS .......................................................................................... 31
Research Questions ......................................................................................................................................................... 31
RQ2: Do university mobile applications employ best practices related to content and services
offered? ................................................................................................................................................................................. 32
Methods ................................................................................................................................................................................ 36
CHAPTER 4 RESULTS .......................................................................................................................................................... 46
Quantitative Analysis and Six Research Questions ........................................................................................... 46
Rankings and Exemplary Applications ...................................................... Error! Bookmark not defined.
Descriptive Qualitative Analysis .................................................................. Error! Bookmark not defined.
CHAPTER 5 DISCUSSION ................................................................................................................................................... 80
Summary .............................................................................................................................................................................. 80
Limitations .......................................................................................................................................................................... 84
Future Research ............................................................................................................................................................... 86
APPENDIX 1A VISUAL DESIGN CODE SHEET ........................................................................................................... 97
APPENDIX 1B INTERFACE AND NAVIGATION DESIGN CODE SHEET .......................................................... 99
APPENDIX 1C INTERACTION DESIGN AND INFORMATION ARCHITECTURE CODE SHEET ............ 101
APPENDIX 1D CONTENT AND SERVICES OFFERED CODE SHEET ............................................................... 103
APPENDIX 1E FUNCTIONAL AND TECHNICAL SPECIFICATIONS CODE SHEET .................................... 105
APPENDIX 2A RANKINGS FOR VISUAL DESIGN ................................................................................................... 107
APPENDIX 2B RANKINGS FOR INTERFACE AND NAVIGATION DESIGN ................................................... 108
APPENDIX 2C RANKINGS FOR INTERACTION DESIGN AND INFORMATION ARCHITECTURE ....... 109
APPENDIX 2D RANKINGS FOR CONTENT AND SERVICES OFFERED ......................................................... 110
APPENDIX 2E RANKINGS FOR FUNCTIONAL AND TECHNICAL SPECIFICATIONS ............................... 111

vii
LIST OF TABLES
Table 2.1: Elements of User Experience – Prevalence in Literature ...........................................12
Table 2.2: User Experience Elements and University Mobile Application Best Practices .........29
Table 3.1: 50 Largest Land-Grant Universities (Total Enrollment) ............................................39
Table 3.2: 30 Land-Grant Universities (Simple Random Selection)...........................................40
Table 4.1: Descriptive Statistics for Functional and Technical Specifications (Category E)......46
Table 4.2: Descriptive Statistics for Content and Services Offered (Category D)......................48
Table 4.3: Descriptive Statistics for Interaction Design and Information Architecture (Category
C)..................................................................................................................................................49
Table 4.4: Descriptive Statistics for User Interface and Navigation Design (Category B) .........51
Table 5.1: App Type and Developer or Vendor ..........................................................................83

viii
LIST OF FIGURES
Figure 2.1: Mobile application user experience workflow..........................................................14
Figure 4.1: University of New Hampshire portrait and landscape orientations ..........................61
Figure 4.2: University of New Hampshire mobile application interface consistency.................62
Figure 4.3: University of New Hampshire mobile application hybrid mobile web integration ..63
Figure 4.4: University of New Hampshire computer labs...........................................................64
Figure 4.5: University of New Hampshire courses......................................................................65
Figure 4.6: University of Arizona user interface and visual design ............................................66
Figure 4.7: University of Arizona mobile websites within mobile app.......................................67
Figure 4.8: University of Arizona font size and audience options ..............................................68
Figure 4.9: University of Arizona share, bookmark, calendar, and view in map options ...........68
Figure 4.10: Ohio State University application home page. This figure illustrates screen shots
that depict the continuity of the continuous scroll capabilities of the OSU homepage and the
additional options in the “Near your current location” section of the homepage........................70
Figure 4.11: University of Florida home screen. This figure illustrates the continuous scroll
feature of the home screen depicted in three separate screen shots.............................................71
Figure 4.12: University of Florida mobile websites ....................................................................72
Figure 4.13: University of Florida customization screens...........................................................73
Figure 4.14: Washington State University photography..............................................................74
Figure 4.15: Washington State University blurred photo effect..................................................75
Figure 4.16: Washington State University navigation and visual hierarchy ...............................76
Figure 4.17: Michigan State University app home, advertising, and navigation screens. The
navigation screenshot depicts a sliding side menu, where the content is pushed right to reveal
the navigation, when the three white line icon is pressed............................................................77
Figure 4.18: Michigan State University places section. These screen shots represent three levels
of information within the places section of the app.....................................................................78
Figure 4.19: Michigan State University food section. These screen shots represent three levels
of information within the food section of the app........................................................................78

1
CHAPTER 1
INTRODUCTION
Statement of the problem
Mobile communication, while not a new phenomenon, has in recent years become an
increasingly important topic for universities. With the outpouring of a variety of web capable
mobile devices and a wave of incoming students that have grown up using smartphones, it is
necessary for universities to develop a mobile presence.
The facts show that students, as well as faculty and staff, are utilizing mobile devices at a
substantial rate (Jones, Ramanau, Cross, & Healing, 2010; Madden, Lenhart, Duggan, Cortesi, &
Gasser, 2013; Rainie, 2013; Amini et al., 2013; Johnson, Means, & Khey, 2013). While much
research exists to provide statistics and facts about mobile usage, very little research exists about
how universities should approach mobile communication and the development of mobile
applications/mobile web. Knowing the current best practices in mobile development and
determining if and how current universities are applying the user experience approach to mobile
development will aid universities in the growth of their mobile strategy.
Rationale for the Study
As many universities begin to build their mobile initiatives or advance those that are
already in place, this research will provide professionals with the background knowledge they
will need to build an effective mobile communication strategy. Simply providing a
comprehensive list of the best practices for mobile applications will be of benefit to universities.
Taking this a step further and evaluating current university mobile applications with an eye
toward – best practices and the user experience approach - will be beneficial for universities

2
going forward with mobile application development. This thesis begins with an assessment of the
different types of mobile applications, which will be followed by an evaluation of why
universities should establish or re-develop a mobile strategy and how they can do so.
Mobile Applications
According to Klein (2012), “A mobile application is a software application that works on
a specific mobile device’s operating system and is downloaded to the device to perform a
specific set of functions.” One of the great debates today in the mobile universe is whether to
develop a mobile application or a mobile website. So why create a mobile application?
McWherter and Gowell (2012, p. 17) state that while the decision of whether or not to create a
mobile app is difficult, a mobile app helps create brand awareness and improves interactions
with customers. One mobile analytics company Flurry, performed a study in June 2011, in
which findings revealed that time spent using mobile applications surpassed time spent using
mobile browsers in the United States (McWherter & Gowell, 2012, p. 17).
Mobile applications are categorized in various ways by different researchers. Some
categorize mobile applications as simply mobile websites and native mobile applications
(Sambasivan, Jon, Udayakumar, and Gupta, 2011; Heitkotter, Hanschke, and Majchrzak, 2012;
Mikkonen & Taivalsaari, 2011). Others have added a third category to the mix, which has been
classified as hybrid apps (Jobe, 2013; Budiu, 2013; Hellbom, 2013; Serrano, Hernantes, &
Gallardo, 2013, Christ, 2011). Determining which type of mobile application to implement --
native, mobile web, or hybrid -- can be challenging. Understanding the nature of each
application type, as well as the advantages and disadvantages of each is essential for the
successful implementation of a mobile app.

3
Native Mobile Apps
Native mobile applications are installed onto the mobile device from an application store
(Google Play or Apple’s App Store), live on the device and are accessible from the device’s
home screen (Budiu, 2013). Jobe (2013) adds that, native apps are written and designed for one
specific operating system, such as, Google’s Android, Apple’s iOS, and Windows, which are
considered the three leading operating systems. When developing a native mobile application, it
is paramount to know which devices to target, have the ability to test and certify the app, as well
as have a strategy to distribute the app to the appropriate users (Fling, 2009, p. 78).
Mobile Websites
According to Budiu (2013) “Web apps [or mobile websites] are not real applications;
they are really websites that, in many ways, look and feel like native applications, but are not
implemented as such” (p. 1). Fling (2009) adds that mobile web apps do not require installation,
but rather use HTML, or more recently HTML5, CSS, and JavaScript to create that “native app
like feel” and can be accessed and run through any mobile web browser (p. 75). Interestingly
mobile web apps, while accessed like any other web page, have the capacity for users to “install”
them on their device by creating bookmarks (Budiu, 2013, p. 1). Mobile websites became
extremely popular with the rise of HTML5, which enables native-like-functionality in a browser,
which eliminates the need to develop device specific native apps (Budiu, 2013, p. 1).
Hybrid Apps
Hellbom (2013) defines hybrid apps as being, “mobile web apps placed in a native
container that enable them to access the devices native functionality” (p. 14). Christ (2011) adds
that hybrid apps combine the benefits of both native and web apps. To create a hybrid app,
developers utilize a platform product such as Adobe’s PhoneGap that enables developers to

4
create an app using HTML, CSS and JavaScript that is compatible with a variety for mobile
devices (Traeg, 2014). Once an application is developed the HTML, CSS and JavaScript files
are bundled into platform-specific deployment packages (Traeg, 2014). PhoneGap, which is
open-source, is not the only platform utilized in the mobile industry to develop hybrid apps.
Christ (2011) identifies three other platforms - Titanium, appMobi, and rhomobile – as popular
for developing hybrid applications (p. 30). Despite hybrid apps combining the best of native and
web, Serrano et al. (2013) warn about always considering the risks that come with the lack of
control over the platform.
It is exceedingly important for large entities – colleges and universities for example – to
develop a mobile strategy due to the overwhelming number of target populations and services
offered. Determining what works and what does not, and following the best practices for mobile
applications, will allow universities to create a strong mobile presence. Developing a mobile
approach can be expensive and cumbersome, as is evident in the evaluation of the different types
of mobile applications, so why should universities go mobile?
Universities and Mobile: The why and how of going mobile
Why Go Mobile?
According to the Nielsen Company, “Today’s consumer is more connected than ever,
with more access to and deeper engagement with content and brands, thanks to the proliferation
of digital devices and platforms” (2014, p. 2). The Nielsen Company and the Pew Research
Center’s Internet and American Life Project are two of the most authoritative sources about
technology, the Internet, and now mobile. Both companies perform extensive research every
year to provide the latest information on technological trends and statistics.

5
Based on a nationally representative survey of 802 teens ages 12-17 and their parents --
performed by the Pew Research Center -- 78% of teens have cell phones, with nearly 47%
owning smartphones (Madden, Lenhart, Duggan, Cortesi, & Gasser , 2013, p. 2). Findings also
revealed that, of those that stated they were internet users, one in four teens say they prefer to go
online on their phones rather than on a desktop or laptop computer (p. 2). In another Pew Study,
Rainie (2013) found that cell phone ownership has hit 91% of adults and of those adults 56%
have smartphones. According to Amini et al. (2013), the same trends are occurring within higher
education:
In 2011, 55% of students surveyed owned a smartphone and 33% deemed smartphones as
extremely valuable for academic success, using them to e-mail professors (66%), check
grades (62%), search the Internet (59%), and communicate with other students about
coursework (61%) (Amini et al., 2013, p. 4).
Texas A&M University performed its own study to determine the use of mobile
technologies on campus and found that 95% of students attending Texas A&M own a cell phone,
81% own a smartphone and 93% used their phone to access the Internet (Ritter, 2012, p. 6).
Ritter (2012, p. 7) also found that 92% of students owning a mobile phone said they used apps.
In another mobile device survey, at the University of Florida from 2010-2011, researchers
looked at ownership and usage of mobile devices for students, faculty, and staff. Results
revealed that 87.2% own a mobile device that has access to the Internet, 27.9% had iPhone’s and
16.3% had Android/Google devices, while 71.9% said they use the Internet on their mobile
device (Johnson, Means, & Khey, 2013, pp. 4-6). Incoming university students don’t simply
have a desire for mobile computing, but have an expectation that it is available and conducive to
their needs (Amini et al., 2013, p. 4).
The incoming generation of college students, who have grown up with computers,
mobile devices, and the Internet, have an acquired aptitude for using new technologies (Jones,

6
Ramanau, Cross, & Healing, 2010, p. 722). It would seem that this generation, often referred to
as digital natives, has extreme confidence in their ability to use smartphones and mobile
technology. In a study at Purdue University, Bowen and Pistilli (2012), asked students to
characterize their level of smartphone usage as one of the following: (1) Novice – using a
smartphone for calls, texting, and e-mail for less than six months, and having a few apps
installed, (2) Intermediate – using a smartphone for more than six months and installing apps in
the event of need or friend recommendation, (3) Advanced – using a smartphone for two years,
using a variety of apps, evaluating differences in similar apps, and recommending apps to
friends, and (4) Expert – I have developed my own mobile apps. The study found that 42% of
students identified as Intermediate and 43% classified themselves as advanced (Bowen and
Pistilli, 2012, p. 4). As the digital native generation continues to proliferate higher education
institutions, due to their confidence in and desire for mobile technology, universities will have to
focus their efforts on going mobile. In a survey conducted by Johnson, Means and Kay (2013),
in 2010-2011, at the University of Florida results showed that:
Most campuses provide a variety of services, from registration to fee payment to course
activities to news about campus events, that students, faculty, and staff desire to access
through mobile devices. As a result, it is increasingly likely that mobile access to
services will become a decision point that will help prospective students and employees
decide where to attend, where to work, and how to rank an institution. Therefore, it is
increasingly important for institutions to “go mobile” (p. 11).
How to Go Mobile
Dixon (2012), discusses how it is important to determine the available options to
implement technology effectively. For app development Dixon (2012) identifies two options at
different ends of the spectrum: (1) “write it yourself options” which provide great flexibility but
are costly to develop, and (2) “free, out of the box options”, often referred to as vendor or white
label apps, that can be implemented quickly but offer limited flexibility. Custom mobile apps

7
allow the freedom to define all the features as well as the global structure of the application.
Maintaining a custom mobile app is expensive and most likely the job will have to be redone due
to advances in technology. White label apps offer already developed and tested features along
with a proven app structure. These apps offer a significant amount of flexibility with regard to
branding and customizing (Dixon, 2012).
In a 2012 survey of 100 Universities in the U.S., Bishop (2012) found that of the
universities with a mobile presence (90%), 32% were using an outside vendor, 49% were
developing in house, and 19% were using an open source platform to implement their mobile
apps. Some of the most popular and highly regarded vendors and producers of white label apps
for higher education are Modo Labs, Ellucian, Blackboard Mobile, and campusM.
Overview of the Study
While the battle between native apps and mobile websites, as well as between custom and
white label apps continues, there are certain best practices that are essential to explore and that
pertain to any type of mobile app regardless of how it is developed. This thesis is divided into
four additional chapters. Chapter 2, the literature review, will interpret best practice methodology
and provide a representation of the current best practices for mobile apps. An evaluation of the
concept of User Experience (UX) will follow, and the best practices will be categorized based on
a User Experience approach. Chapter 3 will identify the methods used for this mixed methods
study. A code sheet based on the UX categories and best practices was developed to evaluate
current university mobile applications from a user experience perspective. Chapter 4 will include
a discussion of the content analysis findings and a descriptive qualitative analysis of the top apps,
to provide further insight into the nature of university mobile application development and
design.

8
CHAPTER 2
LITERATURE REVIEW
Best Practices Methodology
Understanding proven best practices within the mobile industry as well as best practices
specific to mobile in higher education, will allow universities to meet the needs of students,
faculty, and staff, while addressing concerns about “IT infrastructure, planning and governance,
security practices, support strategies, teaching and learning, and fiscal implications” (Dahlstrom
& diFilipo, 2013, p. 4).
Although there is no unifying theory surrounding best practices, there has been some
research that speaks to the methodology of the concept through the lens of applied fields such as
public administration. Over forty years ago, Simon (1963) noted in his classic work, Sciences of
the Artificial, that many applied fields shared a common concern over design and action.
Bretschneider, Marc-Aurele, and Wu (2005) concluded that many of the same applied fields,
such as medicine, management, computer science, and law, are now “concerned with how to
improve actual performance through identification and codification of something typically
referred to as a best practice” (p. 307). Bretschneider et al. (2005) adds that, “while most of the
so called best practices literature comes from management consultants and practicing managers,
it has also found its way into a variety of more academic settings” (p. 307).
It is evident that this concept of best practices came out of a need to understand what
worked and what did not. What convolutes the concept of best practices is the lack of consensus
on a definition. Vesely (2011) discusses how best practices research (BPR) uses the terms “best

9
practice”, “good practice” and “smart practices” interchangeably (p. 100). As stated in “Best
Practices”, in the Encyclopedia of Management (2012):
In a general sense, the term best practice refers to the most efficient way of doing
something. The fastest method that uses the least resources (including labor and parts) to
create the highest quality output is the “best practice.” Almost every industry has
adopted best practices in some aspect of its processes, but those that have made use of it
successfully and publicly have typically done so in the fields of technology development,
quality control, project management, education, manufacturing, health care, and sales
(“Best Practices”, p. 56).
Vesley (2011) explains that, “the primary goal of BPR [Best Practice Research] is to improve the
working of a social institution, typically a business or a non-profit organization, by adopting
certain principles of the working of another institution that appears more successful” (p. 103).
Bretschneider et al. (2005), consider a best practice as the best option or action when compared
to any alternatives when attempting to achieve a deliberative end (p. 309). Burton (2011)
identifies best practices as stemming from the concept of benchmarking, in that when a process
is uniform enough a best practice can be distinguished and accepted more or less “as is” by
another entity (p. 115).
According to Stenström and Laine (2006) definitions of good practice are rather
ambiguous due to the reliance on people, organizations, and the context in which the practice is
identified. While the concept of good practice has been used inconsistently and employed
differently, there are some commonalities surrounding the concept. “Good practice is the
continual process of learning about, feedback concerning, reflection on and analysis of what
works and why something, on the other hand, does not work” (Stenström & Laine, 2006, p. 12).
A comprehensive definition for best practices is: A set of criteria, determined through research,
specific to an industry that can be used by different organizations to measure the effectiveness of
their own procedures or products in order to analyze and improve upon performance.

10
According to Blackboard Mobile (2011),
When it comes to mobile, every organization has unique long-term needs, community
expectations, and user requirements that extend beyond a specific technology or
implementation. A mobile strategy is essential to the creation of an experience that
reflects the priorities, values, and vision of your organization (para. 1).
To develop a strategy it is increasingly important to know how mobile apps and more
specifically university mobile apps are developed most efficiently. In other words, evaluating
the best practices for mobile application development and use will allow universities to create a
successful mobile strategy. General organizations work specifically to determine “best
practices” related directly to mobile. The World Wide Web Consortium, more commonly
known as W3C, is an international community that works to develop web standards, and web
application standards for mobile (World Wide Web Consortium, 2014). The App Quality
Alliance (AQUA) is nonprofit group that works to develop and improve mobile application
quality standards (App Quality Alliance, 2014). EDUCAUSE is a nonprofit association that aids
IT professionals in transforming and advancing technology in higher education. EDUCAUSE
(2014) has over the past few years done research specific to mobile communication development
within higher education.
Mobile and mobile applications are relatively new topics within the research arena, and
therefore little scholarly research exists about “best practices” for mobile applications and
university mobile applications. Organizations like AQUA, W3C and EDUCAUSE, as well as
various bloggers, white app mobile developers, and various whitepapers provide a conceptual
foundation for evaluating the best practices or suggested practices of mobile applications and
university mobile applications. Most of the best practices identified through this research can be
categorized through a user experience perspective and will be identified in the following chapter.

11
User Experience (UX)
User Experience (UX) is a concept that has become an integral piece of the web and
mobile industries in recent years. Similar to the concept of best practices, there is no one defining
theory or concept pertaining to user experience. In the last decade researchers have begun to
really define the concept and build practical models that break the concept down and provide a
means to measure the user experience. Usability, user interface, information design, visual
design, information architecture, interaction design, accessibility, desirability, credibility,
navigation, functionality, control, feedback, consistency, error prevention, content and services
offered, technical aspects, affect, user value, efficiency, effectiveness, and satisfaction are
common terms often identified as being a part of the user experience (Morville, 2004; Garrett,
2011; Al-Khalifa, 2014; Park, Han, Kim, Oh, & Moon, 2011; Psomas, 2007).
According to Nielson and Norman (2014) “User experience encompasses all aspects of
the end-user’s interaction with the company, its services, and its products.” The International
Organization for Standardization (ISO) defines user experience in ISO 9241-110 (2010), a multi-
part standard that addresses human-centered design for interactive systems, as “a person’s
perceptions and responses that result from the use and/or anticipated use of a product, system or
service.” According to Harvey (2013), when discussing user experience software developers and
web designers will often use the following related terms: (1) user-centered design, (2) user
interface (UI) or graphical user interface (GUI), (3) usability, (4) human factors and ergonomics,
and (5) human-computer interaction. Just as the ISO (2010) and Nielsen and Norman (2014)
definitions indicate, user experience encompasses all of these concepts. “The goal of user
experience design in industry is to improve customer satisfaction and loyalty through the utility,

12
ease of use, and pleasure provided in the interaction with a product” (Kujala, Roto, Väänänen-
Vainio-Mattila, Karapanos, & Sinnelä, 2011, abstract).
Research shows that the broad concept of user experience design can be broken down
into seven distinct categories: (1) user needs or user goals (usability), (2) functional and technical
specifications, (3) content strategy, (4) interaction design, (5) information architecture, (6) user
interface, information design, and navigation design, and (7) visual design. Table 2.1 shows the
user experience elements or categories identified through research and their prevalence.
Table 2.1: Elements of User Experience – Prevalence in Literature
Elements of the User Experience
Park et al. 2013
Garrett, 2011
U.S. Department of
Health & Human
Services, 2014
Psomas, 2009
Harvey, 2013
Design, n.d.
Stern, 2014
Lew & Olsina, 2013
DigitalGov, 2014
Cerejo, 2012
Visual Design X X X X X X X X X X
User Interface & Navigation
Design
X X X X X X X
Interaction Design X X X X X X X
Information Architecture X X X X X X X X X
Functional/Technical
Specifications
X X X X X
Content Strategy X X X X X X X
User Needs/ User Goals/ Usability X X X X X X X X X X
As an alternative approach, a model proposed by Garrett (2011) will be used as the
conceptual framework for this study. Garrett (2011, p. 29) classifies the above elements of user
experience into five levels: (1) The Surface Level (visual/sensory design), (2) The Skeleton
Level (interface design and navigation design), (3) The Structure Level (interaction design and
information architecture), (4) The Scope Level (functional specifications and content
requirements), and (5) The Strategy Level (user needs and product objectives). Combining these

13
five levels, the user(s), the developer or organization, best practices, as well as user feedback, --
all of which are key constituents in the user experience -- a model has been created that reflects a
continuous flow between all of these entities (Figure 2.1). A developer would enter the user
experience flow from the strategy level and continue up to the surface level and have a
completed mobile application. The user would then enter the workflow and start their use or
evaluation of the application at the surface level and continue down to the strategy level, at
which point they would be able to determine if the application met their needs, which will in turn
determine if they will be a future and repeated user of the application. The developer would then
want to elicit feedback from the user(s) in order to step back into the process and make any
necessary revisions that will advance their strategy and product to garner more use and meet user
needs. The following sections will define each of the levels and elements of user experience
design, in order to classify the mobile application best practices in the corresponding UX
category. Since we are taking a developer perspective, the next section will begin with the most
abstract user experience concept, user needs and product objectives and continue up the levels to
the most concrete user experience element of visual or sensory design.

14
Figure 2.1: Mobile application user experience workflow
User Experience Approach
Future &
Repeated
Use
User
Feedback
DEVELOPER
Concrete
Abstract
Best Practices
Visual Design Category
Surface
Level
Best Practices
User Interface & Navigation Design Category
Skeleton
Level

Best Practices
Interaction Design & Information Architecture Category
Structure
Level
Best Practices
Functional & Technical Specifications Category
Content Strategy Category
Scope
Level
Usability Testing
User Needs & Site/App Objectives Category
Strategy
Level
USER
Mobile
App

15
Strategy Level
User Needs/Goals and Product Objectives
As a foundation for the model, Garrett (2011) classifies user needs and goals as being a
part of an organization’s strategy.
The foundation of successful user experience is a clearly articulated strategy. Knowing
both what we want the product to accomplish for our organization and what we want it to
accomplish for our users informs the decisions we have to make about every aspect of the
user experience (Garrett, 2011, p. 35).
Stern (2014) takes an in-depth look at user goals and breaks the concept into five layers: (1) user
types, (2) user needs, (3) user motivations, (4) user behaviors, and (5) user outcomes. User types
looks at who the different users might be and where, when, and how they will use a product
(Stern, 2014). Garrett (2011) states that user segmentation is a great way to divide the audience
into more manageable, smaller groups that share key characteristics (p. 42). Identifying user
needs, such as, personal progression, mastery, belonging, and expression are important to
determining how a product can fulfill the user’s goals. Next it is necessary to understand what
motivates users to fulfill their needs. Understanding user’s current behaviors will result in a
greater likelihood of users “developing new habits and loyalty to a product and brand…The
combination of needs, motivations, and behaviors can then translate into meaningful and
measureable outcomes for users” (Stern, 2014). User research is integral to determining user
types, needs, motivations, behaviors, and outcomes. There are a variety of approaches to user
research that can be used individually or collaboratively to identify user goals. Garrett (2011)
identifies the following as substantiated user research methods: (1) market research methods (i.e.
surveys and focus groups), (2) contextual inquiry (i.e. “observing users in the context of their
everyday lives”), and (3) user testing, in which users are observed testing an actual product. User
testing is often referred to as usability testing, which “seeks to make products easier to use” and

16
is often applied in a web context (pp. 46-48). Once user and business needs or in other words a
strategy are established, Garrett (2011) identifies scope as being the next essential element to the
user experience (p. 29).
Scope Level
“Strategy becomes scope when you translate user needs and product objectives into
specific requirements for what content and functionality the product will offer to users” (Garrett,
2011, p. 57).
Functional/Technical Specifications
Cerejo (2012) defined functionality in a mobile user experience context as the tools and
features that enable a user to complete tasks and achieve their goals. Cerejo (2012) provides a
few key guidelines for implementing functionality into a mobile application. A mobile
application should “offer relevant mobile-only functionality (like barcode scanning and image
recognition)” and “ensure that fundamental features and content are optimized for mobile. For
example, make the phone numbers click-to-call” (Cerejo, 2012). Aside from providing
functional features, in order for users to complete tasks and meet their goals, the application must
work. Mobile application developers must take into consideration the speed at which an app
loads and runs, the capability of an app to save settings when interrupted and/or when returning
to the app, as well as the importance of security and privacy measures (Leggett, 2011; Cerejo,
2012; Al-Khalifa, 2014; App Quality Alliance, 2013). In other words the developer must
understand and implement a variety of technical features to improve functionality.
Content Strategy
According to the U.S. Department of Health and Human Services (2014), “Content
strategy focuses on the planning, creation, delivery, and governance of content.” Content not

17
only refers to the words or text on the screen, but the images and multimedia as well (U.S.
Department of Health & Human Services, 2014; Garrett, 2011). Nielsen (2011) identifies that
users on mobile devices want to know the main points and therefore developers should avoid
adding extra, secondary, fluff, especially since mobile apps should be designed for “quick
information consumption.” Any additional background material should be deferred to secondary
screens for those who have additional time and a desire to find out more about a topic. Cerejo
(2012) adds that mobile applications should present a balanced mix of content types, such as
“product information, social content, instructional and support content, and marketing content.”
Multimedia should be used only when it adds value and supports the goals of the website and the
user’s tasks and users should be given control over multimedia. Content must also be appropriate
for the context of mobile; it should be succinct, useful, up-to-date, consistent, and accessible to
all people (Cerejo, 2012; Al-Khalifa; U.S. Department of Health & Human Services, 2014).
Al-Khalifa (2014) developed a framework for evaluating university mobile websites. One
of the four categories Al-Khalifa identified, as being imperative to evaluating university mobile
websites was content and services offered (p. 173). The following are the services Al-Khalifa
(2014) included in his framework as being imperative on university mobile websites: (1) About
the University, (2) announcements, (3) news, (4) events, (5) calendar, (6) maps, (7) emergency,
(8) bus/shuttle information, (9) contact information, (10) directory of people/staff, (11)
admission, (12) schools/colleges, (13) course catalogue, (14) library, (15) sports/athletics, (16)
alumni, and (17) social media (p. 174). Developers must determine which of these services are
essential to their strategy and incorporate content in those categories appropriately. Some of
these services, such as alumni, sports/athletics, and library involve different audiences and user
needs and it might be beneficial for a university to create multiple mobile applications that are

18
geared specifically toward these audience segmentations. No matter what, developers need to
determine which content items and services are most important to their users and develop the
scope of the mobile application around them. Once the scope, or the functional specifications and
content and services offered are established a developer will need to determine how the pieces fit
together into a cohesive mobile application (Garrett, 2011, p. 79).
Structure Level
Garrett (2011) identifies the structure level as the next element in the user experience.
“The structure defines the way in which the various features and functions of the site fit
together” (p. 20-21). Garrett breaks the structure level into interaction design and information
architecture (p. 80).
Interaction Design
Interaction design is defined by Fling (2009) as the way in which a user can participate
directly or indirectly with the application to accomplish goals and create a meaningful
experience (p. 90). Due to interactive nature of mobile devices, especially with the proliferation
of touch screens, mobile developers must consider the ergonomics, system interactions, device
interactions, and mobile-specific interaction patterns in order to build engaging interfaces (U.S.
Department of Health & Human Services, 2014; McVicar, 2013; Stern, 2014).
Ergonomics. According to The International Ergonomics Association (2014)
“Ergonomics is the scientific discipline concerned with the understanding of interactions among
humans and other elements of a system, and the profession that applies theory, principles, data
and methods to design in order to optimize human well-being and overall system performance.”
When considering ergonomics for mobile devices and interaction design, developers must “pay
attention to device dimensions as well as the pragmatic concerns of touch screens” (McVicar,

19
2013). Determining the appropriate size(s) of touch areas to enable accuracy and confidence
when users interact with mobile devices is a form of ergonomics in the mobile industry.
Different mobile devices recommend different sizes for touch areas, though those numbers tend
to range between 44px and 57px on standard screens and 88px to 114px on retina screens
(McVicar, 2013).
System Interaction. Stern (2014) identifies system interaction as consisting of
“navigation, flows, feedback, and notifications to help the user progress and achieve their goals.”
Giving users clues through appearance about how a feature will function, anticipating and
mitigating errors, providing error messages when an error has occurred, providing feedback once
an action is performed, and aiming to provide an immediate response time (less than 0.1 second)
are specific ways in which developers can address interaction at a system level (U.S. Department
of Health & Human Services, 2014; Stern, 2014).
Device Interaction. According to Stern (2014) “When designing for an experience it’s
key to understand the capabilities and constraints of the target devices, including screen sizes,
connectivity, user interface conventions, and other factors.” Gestures, geolocation, audio
recording, camera capabilities, and push notifications are components specific to mobile phones
and these as well as other mobile-specific features should be integrated into mobile applications
to enhance the interactive nature of the app. Gestures are actions that all major touch operating
systems employ that allow users to perform specific tasks. Some of the most common gestures
used in touch technology include: (1) press, which in an iOS platform selects a primary action,
(2) double press, enables zoom in or zoom out functionality, (3) small swipe, will reveal a delete
button by sliding horizontally on a table view, (4) large swipe, allows for scrolling across
content, and (5) pinch/spread, is another option for zooming in and out (McVicar, 2013).

20
Pattern Interaction. Stern (2014) identifies everything from headers and menus to
calendars and maps as being patterns. McVicar (2013) adds that “some of the most valuable
[interaction patterns] include those used to improve navigation, select content, sign in/out, and
negotiate forms.” Typical navigation patterns consist of the expanding menu, the side menu, the
hub and spoke menu, the coverflow, and the flip over/peel back. When an application requires a
username, email, and password, providing users with auto sign in and providing an option for the
system to remember sign in details, will enable create a more positive user experience (McVicar,
2013). Filling out forms can be an extremely frustrating task on a mobile device. Saving or
recalling user details by utilizing a login process and ensuring that the relevant keyboard is
displayed for the task at hand (for example display the number keyboard when asking users to
enter a phone number) are essential for creating effective forms with input capabilities.
Interaction design can make or break a mobile application. Interaction is at the heart of mobile
and mobile applications and it is therefore essential for developers to provide an intuitive
interactive experience to elicit user satisfaction and repeated use.
Information Architecture
Just as interaction design emphasized patterns and sequences, information architecture
deals with the sequences and options involved in presenting content to the user (Garrett, 2011, p.
80-81). Fling (2009) defines information architecture as, “The organization of data within an
informational space. In other words, how the user will get to information or perform tasks within
a website or application” (p. 90). “On content sites, information architecture is concerned with
creating organizational and navigational schemes that allow users to move through site content
efficiently and effectively” (Garrett, 2011, p. 89). McVicar (2012) identifies mobile devices as

21
having distinctive information architecture patterns: (1) hierarchy, (2) hub and spoke, (3) nested
doll, (4) tabbed view, (5) bento box/dashboard, and (6) filtered view.
In a hierarchy pattern, information is structured with a standard index page and a series of
subpages. Hierarchy patterns organize complicated site structures efficiently and are often
associated with desktop architecture (McVicar, 2012). The hub and spoke pattern begins with a
central index or “hub” and users navigate out. In order to reach another “spoke” users must
navigate back to the “hub.” The hub and spoke pattern is the iPhone’s default pattern. This
pattern is ideal for users focused on a specific task, but can become frustrating for users wanting
to multi-task (McVicar, 2012). A nested doll pattern is linear and leads users from general to
more detailed information and provides a perception of moving forward and backward. Nested
doll patterns are “good for apps or sites with singular or closely related topics” and “can also be
used as a sub section pattern inside other parent patterns, such as the standard hierarchy pattern
or hub and spoke” (McVicar, 2012, Mobile Information Architecture: Nested doll section, para.
2). A tabbed view is a group of sections that are linked together through a toolbar or menu.
Tabbed views allow for multi-tasking and quick understanding of the complete functionality of
the app at initial launch (McVicar, 2012). The bento box or dashboard pattern allows for more
detailed content to be displayed directly on the index screen. The pattern is better suited for
tablet devices due to the complexity (McVicar, 2012). Filtered view patterns use filters and
search options that allow users to explore content through alternative views. This pattern is
effective for sites that contain large amounts of articles, images, and videos (McVicar, 2012).
Picking one of the above information architectures is dependent on context (business
goals, culture, politics, technology, and resources), content (objectives, document and data types,
and volume), and users (audience, tasks, needs, behavior, and experience), which is referred to as

22
the “information ecology” (U.S. Department of Health & Human Services, 2014). Cerejo (2012)
identifies the following as key elements of mobile information architecture: (1) links to the main
features and content are presented on the landing page, (2) users are able to navigate in as few
taps as possible to the most important content, (3) navigation is broad and shallow not deep.
Skeleton Level
According to Garrett (2011) the skeleton level identifies aspects of the interface and
navigation, which allows for further refinement of the structure in to something more concrete
(p. 107).
User Interface
The design of the user interface “is all about selecting the right interface elements for the
task the user is trying to accomplish and arranging them on screen in a way that will be readily
understood and easily used” (Garrett, 2011, p. 114). Interface elements include: (1) input
controls – buttons, text fields, checkboxes, radio buttons, dropdown lists, list boxes, toggles, and
date fields, (2) navigational components – breadcrumbs, slider, search field, pagination, tags,
icons, (3) informational components – tooltips, icons, progress bar, and notifications, and (4)
containers – accordion (U.S. Department of Health & Human Services, 2014). When designing
the interface of a mobile application it is paramount that developers create consistency in terms
of action sequences, terms, layouts, and buttons. Navigation should be easy and intuitive, use
native icons consistently, and support standard back button navigation where appropriate. User
Interface elements such as text and graphics should display without distortion, blurring,
pixilation, and aliasing, or in other words have a high level of visual quality. The interface of a
mobile application should also be adaptable to landscape and portrait orientations on a mobile
device (App Quality Alliance, 2013). With all of these complex elements available to

23
developers, the key to a successful interface design is simplicity (U.S. Department of Health &
Human Services, 2014).
Navigation Design
According to Garrett (2011) “Navigation design is the specialized form of interface
design tailored to presenting information spaces” (p. 108). Garrett (2011) identifies navigation
design as consisting of three simultaneous goals that need to be accomplished: (1) navigation
elements must be able to facilitate real user behavior by means of getting users from one point to
another, (2) “the navigation design must communicate the relationship between the elements it
contains”, and (3) “the navigation must communicate the relationship between its contents and
the page the user is currently viewing” (p. 118-119). Cardello (2014) does a great job of
differentiating between navigation, information architecture, and user interface, all of which as
Garret (2011) identified are interrelated as elements of the user experience approach. According
to Cardello,
The information architecture (IA) is not part of the on-screen user interface (UI) – rather,
IA informs UI…Navigation is a collection of user interface components which can
include, global navigation, local navigation, utility navigation, breadcrumbs, filters,
facets, related links, footers, and fat footers (Cardello, 2014, What is website information
architecture section, para. 1, What is website navigation section, para. 1).
Global or main navigation can take many forms in a mobile application including the
following four options. One option is the expanding menu, which utilizes a menu icon (often
three horizontal, stacked lines) housed in the header. When the icon is tapped a list of menu
items is revealed. Another type of navigation structure is the side menu or sliding side menu. An
icon is touched (similar to that of the expanding menu) and a menu expands or slides open to the
side and displays a vertical list of menu options. A side menu will often have categories, as it is
an effective menu for apps with a high volume of menu options. The next navigation option is

24
the tabbed menu. With a tabbed menu there is a static toolbar that is displayed in either the footer
or header. Finally there is the hub and spoke menu option. Hub and spoke utilizes a central home
screen that provides various menu options pointing to different sections. Returning home is
achieved through a back button (McVicar, 2013)
Surface Level
Visual/Sensory Design
According to U.S. Department of Health and Human Services (2014) “Visual design
focuses on the aesthetics of a site [application] and its related materials by strategically
implementing images, colors, fonts, and other elements.” Garrett (2011) identifies visual design
as the aspects that the user will notice first and classifies visual design as belonging to the
surface plane. According to Garrett (2011) “Here, content, functionality, and aesthetics come
together to produce a finished design that fulfills the goals of the other four planes [strategy,
scope, structure, and skeleton]” (p. 141). Specific components of visual design include: (1)
contrast, (2) unity or consistency, (3) typography, (4) visual hierarchy, (5) brand consistency, (6)
texture, shape, and space (Psomas, 2007; U.S. Department of Health & Human Services, 2014;
Garrett, 2011). McVicar (2014) classifies designing for mobile in terms of designing for
communication. In other words, communication can be enhance through visuals and imagery,
and the overall user experience can be improved by providing good visual styling that supports
the purpose of the application.
Contrast. “Contrast between colors on a page both draws attention to page elements and
enhances readability” (Barnes, 2010, p. 171). Contrast is a great tool for creating emphasis and
can be accomplished through the use of different sizes, colors, and direction (U.S. Department of
Health & Human Services, 2014). McVicar (2014) adds that creating contrast between content

25
and functionality will help communicate connections and actions within the interface. “Using a
mix of neutral and bright colors, or colors at opposing ends of the color wheel will ensure that
specific items stand out to the user” (McVicar, 2014). Contrast between headlines, type, buttons,
as well as backgrounds will improve functionality of the interface and improve ease of use for
mobile app users.
Unity or Consistency. Consistency is one of the most important elements of visual
design, as it applies to all other elements including but not limited to color, typography, space,
and visual hierarchy. In fact “unity has to do with all elements on the page visually or
conceptually appearing to belong together. Visual design must strike a balance between unity
and variety to avoid a dull or overwhelming design” (U.S. Department of Health & Human
Services, 2014). Research shows that consistency in visual design leads to fewer errors,
reduction in task completion times, increased user satisfaction, and reduction in learning time
(Galitz, 2007). Creating usable layouts is a great way to establish a sense of unity or consistency
across a mobile application. Grid-based layouts help establish a consistent visual design.
The structure of evenly spaced vertical lines acts as a guide for arranging content. The
grid identifies event spaces, making it easier for designers to determine the most effective
placement for buttons, headlines, or images. Placing these components along a grid helps
to lead users on their journey, while simultaneously creating a clean and aesthetically
pleasing visual (McVicar, 2014, Usable layouts: structure of the layout section, para. 1).
Consistency can make or break a mobile application and applying the concept to all aspects of
mobile application visual design will create a stronger overall user experience.
Typography. The arrangement, style, and appearance of text is referred to in the design
world as typography (Bennett, 2005). “Whether or not a site is readable impacts its credibility”
(Barnes, 2010). Choosing the appropriate size, alignment, color, and spacing of fonts/type are
key elements to an effective visual design. Galitz (2007, p. 169) states that the characteristics of

26
fonts [such as size, alignment and color] are great tools for organizing and identifying the
important screen elements, establishing reading order, and generating specific moods. Research
reveals that when working with large blocks of text it is ideal to keep fonts and text elements
simple. More specifically it is advisable to use familiar fonts such as Arial or Verdana to achieve
the best reading speed and not to use more than two font families, e.g. Arial and Times New
Roman (Galitz, 2007, Garrett, 2010). One of the easiest ways to keep fonts simple, yet establish
differentiation is to use different sizes of the same font (Bennett, 2005). Garrett (2010) adds that
when working with headlines or short labels, such as navigation elements, it is appropriate to use
fonts with “a little more personality” (p. 147). The key to good typography is to eliminate visual
clutter, establish a visual hierarchy through the use of different font families, font sizes, and font
colors.
Visual Hierarchy. According to Barnes (2010) “A visual hierarchy is a group of visual
(and verbal) elements arranged according to emphasis in a layout” (p. 171). Visual hierarchy is
often created through the use of different font sizes, colors, as well as spacing of elements and
content sections on the screen. Creating hierarchy through visual design creates what McVicar
(2014) refers to as vertical rhythm, which aids in the communication and therefore understanding
of content within mobile applications (McVicar, 2014; U.S. Department of Health & Human
Services, 2014). Utilizing a grid system can aid a mobile app in creating visual hierarchy and
vertical rhythm. “The structure of evenly spaced vertical lines acts as a guide for arranging
content. Placing these components [buttons, headlines, or images] along a grid helps to lead
users on their journey, while simultaneously creating a clean and aesthetically pleasing visual”
(McVicar, 2014).

27
Brand Identity. “Enforcing design consistency across media presents your audience –
customers, prospects, shareholders, employees, or casual observers – with a uniform impression
of your brand identity” (Garrett, 2011). Brand identity should be evident throughout the visual
design from navigation elements appearing consistently across all screens, to the color palette, as
well as the use of a logo or other brand identifier (Garrett, 2011).
Texture, Color, Shape, and Space. Lines, shapes, texture, and space are all basic
elements of visual design that can engage users and help build trust and interest in a brand (U.S.
Department of Health & Human Services, 2014). Lines define shapes, create space, and produce
texture. While shapes can be created with lines, they are also defined by variations in color and
texture. Color helps to create depth, emphasis, and aids in the organization of information (U.S.
Department of Health & Human Services, 2014). Texture is often used to enhance detail, which
in turn makes elements of a design appear more realistic. Texture is also used to make empty
space more visually appealing, and often creates a sense of depth when used as backgrounds
(Bennett, 2005). The use of space within visual design works to “reduce noise, increase
readability, and create illusion” (U.S. Department of Health & Human Services, 2014). Space
also plays an important role in the functionality of the mobile interface. “It is essential that users
can explore content without committing to an action that they did not intend, and without
activating an item when they were attempting to scroll. In other words, the spacing between
elements must be sufficient to allow a user to easily navigate through these elements” (McVicar,
2014). The elements of visual design, like space, are not only essential to create an aesthetically
pleasing mobile application, but it is these concrete visual details that the user will see first and
the success of these elements will dictate the entire user experience.

28
Garrett (2011) stated that user experience design “means taking into account every
possibility of every action the user is likely to take and understanding the user’s expectations at
every step of the way through the process.” He also adds that, “By breaking the job of crafting
the user experience down into its component elements, we can better understand the task as a
whole” (p. 19). By evaluating university mobile applications through these component elements
it becomes apparent what best practices associated with the user experience design process are
being utilized effectively and which could be improved upon within university mobile apps.
Table 2.2 provides a comprehensive summary of the mobile application best practices and their
corroborating user experience categories. The five user experience elements that will be
evaluated in this study are listed horizontally across the top of the table. Below the user
experience elements are the university mobile application best practices that will also be assessed
in this study. There are eight to ten best practices that are listed vertically below their respective
user experience element.

29
Table 2.2: User Experience Elements and University Mobile Application Best Practices
USER EXPERIENCE ELEMENTS
Functional &
Technical
Specifications
Content &
Services Offered
Interaction Design
& Information
Architecture
User Interface &
Navigation Design
Visual & Sensory
Design
BEST PRACTICES
Privacy Policy
App provides a
clearly visible link
to the privacy
policy and/or terms
and conditions
Organization
Content and
services offered are
organized
efficiently for easy
navigation (i.e. by
target audience)
Gestures
App employs
gestures that are
consistent with the
operating systems
standards
Page Titles
App provides
descriptive yet
concise page titles
that clue the user
into the content of
the page
Contrast
App uses high
contrast color
schemes (mix of
neutral and bright
colors)
Trustworthy
App provides user
with a sense of
security and is
transparent
(informs users
about what
personal
information is
required, who it
will be shared with,
and why)
Target Audiences
Which of the
following audiences
does the app
target? Select all
that apply (Current
Students,
Prospective
Students, Faculty/
Staff, Campus
Visitors, Donors,
Alumni)
Walkthroughs
Overlay
instructions or
coach marks are
provided upon
initial launch to
show users how to
use different
features of the app
when deemed
necessary
Consistent
Interface Design
The interface of the
app is consistent –
when app directs
users to webpages
they are mobile
friendly and fit with
the app interface
design
Visual Hierarchy
App creates a sense
of visual hierarchy
and flow (i.e.
elements are
arranged according
to emphasis in the
layout – different
font sizes, colors,
spacing elements,
and sections)
Load Time &
Speed
Load time & speed
is acceptable for the
purpose of the app
and does not alter
the user experience
– is controllable
Succinct Copy
Copy is reflective of
the main points that
are ideal for quick
information
consumption.
Additional info is
deferred to
secondary screens
Error Messages
App provides
helpful error
messages when a
task does not go as
planned (i.e.
messages when
login fails)
Orientation
App supports
landscape and
portrait orientation
and is capable of
rapid transition
between
orientations
Typography
Fonts and text
elements are simple
and readable; No
more than 2 fonts
are used throughout
the app
Interruptions
App has the
capability to accept
an incoming call
while running
Logo/Name
The university name
and/or logo appear
on the landing page
or main page
Repeated Use
App encourages
repeated use
through interaction
with features of the
app
Screen Space
Design is for a
single window or
full screen.
Main task is front
and center
Texture & Shape
App uses texture
and/or shape to
enhance detail and
make empty space
more appealing
Continuation
App has the
capability to
resume tasks from
the same place or a
logical restarting
point
Sharing Content
App has easy
access to social
media platforms
and the ability to
share content to
these platforms
Notifications
App notifies user
when it is leaving
the app to access
other phone
features and apps
Graphics/Text
Graphics and text
are not distorted,
blurred or
pixelated
Unity &
Consistency
Colors, space,
typography, and
layout are
consistent and
create unity

30
Opt-In Consent
App obtains opt-in
consent prior to
accessing location
data and/or geo-
tagging with
photos and videos;
provides an option
to opt-out
Services Offered
App offers
services that are
appropriate for a
university mobile
app; services
offered are usable
and enhance the
user experience
Device Features
App enables
interaction with
mobile device
features (camera,
location functions,
and other apps
available on the
device)
Labels & Icons
Labels and Icons
are descriptive,
clear, concise, and
consistent (inform
the user about
what content will
be provided when
clicked)
Brand Identity
The brand of the
university is
consistent
throughout the app
with the use of
university colors,
logos, and/or
brand identifiers
Function
Progress
App provides
visual indication
of activity (i.e.
prompt for user
input, splash
screens, progress
bars, “please
wait” text, etc.)
Multimedia
Images and video
are used
appropriately to
supplement
content and
provide a good
user experience
Input
Default values are
displayed for input
and appropriate
keyboard is
available when
entering data (i.e.
numbers are
available when
asking for a phone
number)
Navigation
Buttons and
navigation are
consistent;
communicates the
relationship
between
navigation
elements and the
content they are
linked to
Color
Color is used to
give prominence to
content items,
demonstrate how
items are
connected, and
helps structure
content (i.e.
separates content)
User Feedback
App provides
users with a
means to contact
the organization
with questions
and/or provide
feedback
Up-to-Date
Content
Content is updated
with current and
accurate
information
Architecture
Pattern
App utilizes a
consistent
architecture
pattern that is
shallow not deep
in nature
Button Size
App uses
appropriately
sized buttons and
touch targets (i.e.
buttons are easy
to click and react
appropriately to
touches)
Aesthetics
Overall design is
aesthetically
pleasing
Click-to-Call
App provides
phone numbers
that can be clicked
and prompt the
user to make a call
Horizontal
Scrolling
App avoids
horizontal
scrolling
Login Security
App asks users to
login
appropriately (i.e.
when accessing
personal info).
Logins are not
overused
Back Button
Navigation
App supports/uses
standard back
button navigation

31
CHAPTER 3
RESEARCH QUESTIONS AND METHODS
Research Questions
The user experience of mobile applications and more specifically university mobile
applications is a complex ecosystem consisting of: user needs or user goals (usability), functional
and technical specifications, content strategy, interaction design, information architecture, user
interface, information design, navigation design, and visual design. Best practice methodology
goes hand-in-hand with user experience, as both these concepts work to achieve an optimum
product. In this case the product is a mobile application for a university, and by establishing the
best practices associated with the elements of user experience, current university mobile
applications can be evaluated to determine current mobile presence at universities as well as
inform future university mobile application development. The following research questions will
help establish a consensus as to how current universities adhere to user experience best practices
for mobile applications.
RQ1: Do university mobile applications achieve functional and technical best practices?
By evaluating university mobile applications based on the following best practices
compiled through research, I was able to determine if and how universities are achieving the
functional and technical elements of user experience.
1. Load Time and Speed: Load time and speed of app is acceptable for the purpose of the
app and does not alter the user experience by being uncontrollable
2. Trustworthiness: App provides user with a sense of security and is transparent (informs
users about what personal information is required, who it will be shared with, and why)
3. Privacy Policy: App provides a clearly visible link to the privacy policy and/or terms and
conditions
4. Interruptions: App has the capability to accept an incoming call while running

32
5. Continuation: App has the capability to resume tasks from the same place or a logical
restarting point
6. Click-to-Call: App provides phone numbers that can be clicked and prompt the user to
make a call
7. Opt-In Consent: App obtains opt-in consent prior to accessing location data and/or geo-
tagging with photos and videos; provides an option to opt-out
8. User Feedback: App provides users with a means to contact the organization with
questions and/or provide feedback about the app
9. Function Progress: App provides visual indication of activity (i.e. prompt for user input,
splash screens, progress bars, “please wait” text, etc.)
10. Login Security: App asks users to login appropriately (i.e. when accessing personal info).
Logins are not overused
Sources: Garrett, 2011; Harvey, 2013; Lew & Olisina, 2013; DigitalGov, 2014; Cerejo,
2012; App Quality Alliance, 2013.
RQ2: Do university mobile applications employ best practices related to content and
services offered?
compiled through research, I was able to determine what type of content was most prevalent
throughout university mobile applications (informational, social, instructional support, and
marketing content) and what services universities offer to its users. This evaluation may also be
able to shed light onto the target audience(s) for university applications and whether individual
audience based applications exist and/or should be developed.
1. Target Audiences: Current students, prospective students, faculty/staff, campus visitors,
donors, and alumni
2. Organization: Content and services offered are organized efficiently for easy navigation
3. Succinct Copy: Copy is reflective of the main points that are ideal for quick information
consumption. Additional info is deferred to secondary screens
4. University Logo/Name: The university name and/or logo appear on the landing page or
main page of the app
5. Multimedia: Images and video are used appropriately to supplement content and provide
a good user experience
6. Sharing Content: App has easy access to social media platforms and the ability to share
content to these platforms
7. Up-to-Date Content: Content is updated with current and accurate information

33
8. Services Offered: App offers services that are appropriate for a university mobile app
environment; services offered are usable and enhance the user experience
9. Which services are available on the mobile application?
Sources: Garrett, 2011; U.S. Department of Health & Human Services, 2014; Al-Khalifa,
2014; Stern, 2014; Lew & Olsina, 2013; DigitalGov, 2014; Cerejo, 2012; App Quality
Alliance, 2013.
RQ3: How are interaction design and information architecture achieved within university
mobile applications?
Evaluation of the best practices associated with interaction design and information
architecture allowed for an analysis of how universities were employing interaction design and
information architecture into their mobile applications. Elements such as gestures, auto sign-in,
input, error messages, repeated use, and architecture patterns were measured.
1. Gestures: App employs gestures that are consistent with the operating systems standards
2. Walkthroughs: Overlay instructions or coach marks are provided upon initial launch to
show users how to use different features of the app when deemed necessary
3. Error Messages: App provides helpful error messages when a task does not go as planned
(i.e. messages when login fails)
4. Input: Default values are displayed for input and appropriate keyboard is available when
entering data (i.e. numbers are available when asking for a phone number)
5. Repeated Use: App encourages repeated use through interaction with features of the app
6. Device Features: App enables interaction with mobile device features (camera, location
functions, and other apps available on the device)
7. Notifications: App notifies user when it is leaving the app to access other phone features
and/or apps
8. Architecture Pattern: App utilizes a consistent architecture pattern that is shallow not
deep in nature (i.e. hierarchy, hub and spoke, tabbed view, nested doll)
9. Which information architecture patterns are used; hierarchy, hub and spoke, nested doll,
tabbed view?
Sources: App Quality Alliance, 2013; Nielsen, 2011; Cerejo, 2012; McVicar, 2012 & 2013;
Psomas, 2009; Harvey, 2013; Stern, 2014; Lew & Olsina, 2013; DigitalGov, 2014.

34
RQ4: Which interface and navigation design best practices are prevalent throughout
university mobile applications?
compiled through research, I was able to determine if and how universities are utilizing the user
experience elements of interface and navigation design.
1. Horizontal Scrolling: App avoids horizontal scrolling
2. Graphics/Text: Graphics and text are not distorted, blurred or pixelated
3. Orientation: App supports landscape and portrait orientation and is capable of rapid
transition between orientations
4. Screen Space: Design is for a single window or full screen; main task is front and center
5. Consistent Interface Design: The interface of the app is consistent – when app directs
users to webpages they are mobile friendly and fit with the app interface design
6. Button Size: App uses appropriately sized buttons and touch targets (i.e. buttons are easy
to click and react appropriately to touches)
7. Back Button Navigation: App supports/uses standard back button navigation
8. Labels and Icons: Labels and Icons are descriptive, clear, concise, and consistent (inform
the user about what content will be provided when clicked)
9. Page Titles: App provides descriptive yet concise page titles that clue the user into the
content of the page
10. Main navigation utilizes a navigation format consistently (expanding menu, side menu,
tabbed menu, hub and spoke menu)
Sources: App Quality Alliance, 2013; Nielsen, 2011; Cerejo, 2012; McVicar, 2012; Garrett,
2011; U.S. Department of Health & Human Services, 2014; Psomas, 2009; Harvey, 2013.
RQ5: Do university mobile applications utilize visual design best practices?
compiled through research, I was able to determine how universities are applying the user
experience element of visual design to their apps.
1. Contrast: App uses high contrast color schemes (mix of neutral and bright colors)
2. Unity and Consistency: Colors, typography, space, and layout are designed consistently
and create a sense of unity throughout the app
3. Typography: Fonts and text elements are simple and readable; No more than 2 fonts are
used throughout the app

35
4. Visual Hierarchy: App creates a sense of visual hierarchy and flow (i.e. elements are
arranged according to emphasis in the layout – through the use of different font sizes,
colors, spacing elements, and content sections)
5. Brand Identity: The brand of the university is consistent throughout the app with the use
of university colors, logos, and/or brand identifiers (i.e. slogans)
6. Texture and Shape: App uses texture and/or shape to enhance detail and make empty
space more appealing
7. Aesthetics: The overall design of images, colors, fonts, and other elements are visually
and sensually pleasing; design enhances user engagement and interest
8. Color: Color is used to give prominence to content items, demonstrate how items are
connected, and helps structure content (i.e. separates content)
Sources: App Quality Alliance, 2013; Al-Khalifa, 2014; World Wide Web Consortium, 2008;
McVicar, 2014; Garrett, 2011
RQ6: Overall, what were the most prevalent and least evident best practices?
Subsequent to collecting data, the researcher attempted to analyze the reliability of the
dichotomous variables and interval measures with the idea of combining them into a reliable
summary index for the five UX categories. However, when the Cronbach reliability analysis was
conducted only a couple of measures proved satisfactorily reliable and as a result no indices were
used. Instead, percentage and mean scores from the five user experience design categories were
combined into summary tables, in order to determine the most prevalent and least evident best
practices among all categories.
RQ7: Which university mobile applications were ranked highest based on prevalence of
best practices and what made them exemplary?
Once each app was evaluated, the use of descriptive and non-parametric statistics was
used to rank the applications based on their use of best practices. A descriptive qualitative
analysis revealed which best practices these top apps used effectively and what made the apps
exemplary with regard to best practices and user experience design.

36
Methods
To investigate these questions a content analysis was performed of the primary university
native application, for 30 of the 50 largest land-grant universities in the United States.
Content Analysis
According to Stempel (2003) “Content analysis is a formal system for doing something
we all do informally rather frequently – draw conclusions from observations of content” (p. 209).
The history of modern content analysis dates back to World War II, when researchers performed
studies in which they counted words to verify document authenticity and used content analysis to
determine changes in troop concentration across occupied Europe. The most notable use of
content analysis began in the 1950s when researchers began using it as a tool for media analysis
of newspapers and radio (p. 150). According to Busch et al. (2012) when performing a content
analysis, “researchers quantify and analyze the presence, meanings and relationships of…words
and concepts, then make inferences about the messages within the texts, the writer(s), the
audience, and even the culture and time of which these are a part.” Understanding that content
analysis is systematic, objective, and quantitative aids in researchers’ capability to perform a
content analysis competently and efficiently (Stempel, 2003; Wimmer & Dominick, 2006).
Stempel states that:
Systematic means, first that a set procedure is applied to all content being analyzed.
Second, it means that categories are set up so that all relevant content is analyzed.
Finally, it means the analysis is designed to secure data relevant to a research question or
hypothesis (Stempel, 2003, p. 210).
“Objectivity is achieved by having the categories of analysis defined so precisely that different
persons can apply them to the same content and get the same results” (Stempel, 2003, p. 210).
“Quantitative means simply the recording of numerical values or frequencies with which the
various defined types of content occur” (Stempel, 2003, p. 210).

37
Content analysis was selected to evaluate the best practices and corresponding user
experience categories. The systematic component of content analysis was met through the use of
a coding sheet and categories of user experience and best practices were defined precisely to
achieve objectivity. The study met the third content analysis criteria by counting the frequency
and prevalence of the best practices and user experience elements of the 30 land-grant university
mobile applications. Content analysis was the research method of choice due to its ability to
glean great insight from what has already been done. Other research approaches that would have
been appropriate for this study include in-depth interviews, surveys, or focus groups with either
developers or students at universities. Since the goal of this study is to determine whether
universities utilize the user experience best practices and not to gather information regarding user
or developer perspectives a content analysis was deemed the most appropriate form of research.
Sampling and Selection of Apps
According to the Association of Public and Land-Grant Universities (2012) “A land-grant
college or university is an institution that has been designated by its state legislature or Congress
to receive the benefits of the Morrill Acts of 1862 and 1890” (What Section, para. 1). There are
currently a total of 111 land-grant universities designated by the United State Department of
Agriculture (2014), which are located across all 50 States, as well as the American Samoa,
Guam, Micronesia, Northern Marianas, Puerto Rico, and the Virgin Islands (p. 2). The study
selected the 50 largest land-grant universities in the United States and randomly selected 30 of
the 50 largest to use in the study.
The 50 largest land-grant universities were determined based on total enrollment numbers
and excluded the University of California System since there is not a main campus. To select the
30 universities an online random number generator was used, in which 30 numbers from 1 to 50

38
were selected. Table 3.1 shows the 50 largest land-grant universities and each university’s
corresponding number 1 to 50, which is based on enrollment numbers, with 1 having the largest
enrollment and 50 the lowest enrollment. If a selected university was found to not have a mobile
application, another university was selected at random from the remaining list of 20 universities.
Random Sample
According to Trochim (2006) when researchers use some form of random selection they
are performing probability sampling (Probability Sampling Section, para. 1). Simple random
sampling is the simplest form of random sampling, where the objective is to “select n units out of
N such that each NCn has an equal chance of being selected” (Trochim, 2006, Simple Random
Sampling Section, para. 1). As defined by Trochim (2006) “N = the number of cases in the
sampling frame” while “n = the number of cases in the sample” and “NCn = the number of
combinations (subsets) of n from N” (Some Definitions Section).
Numbers were randomly selected from within the range of 1 to 50 (including numbers 1
and 50) therefore N = 50. Duplicate numbers were not allowed. The following numbers were
automatically generated (n = 30) by an online random number generator: 03 11 01 31 40 39 43
16 22 29 08 21 20 36 42 34 23 15 50 33 35 05 32 37 47 30 17 18 44 02. Out of the sample of 30
land-grant universities, eight schools (North Dakota State University, Montana State University,
Cornell University, Mississippi State University, University of Massachusetts Amherst,
University of Maine, University of Delaware, and West Virginia University) did not have a
designated university mobile application and therefore further random selection was required. In
the end, 20 of the 50 schools did not have a main university application or their app had
insufficient functionality to be evaluated in this study. Table 3.2 displays the final 30 land-grant
universities that were evaluated in this study.

39
Table 3.1: 50 Largest Land-Grant Universities (Total Enrollment)*
1 Ohio State University 64,930 26 University of Kentucky 29,802
2 University of Minnesota 63,929 27 University of Connecticut 28,218
3 University of Florida 56,683 28 Oklahoma State University 27,756
4 Texas A&M University 53,966 29 University of Nebraska 27,233
5 Michigan State University 53,371 30 Kansas State University 27,119
6 Pennsylvania State University 49,223 31 Auburn University 27,046
7 University of Illinois 47,684 32 University of Hawaii Manoa 23,898
8 University of Wisconsin 46,020 33 University of Delaware 23,304
9 Rutgers University 44,809 34 Mississippi State University 22,956
10 Purdue University 44,267 35 Clemson University 22,675
11 University of Maryland 43,352 36 Cornell University 21,626
12 University of Arizona 43,038 37 University of Nevada – Reno 20,947
13 North Carolina State University 40,720 38 University of Rhode Island 19,257
14 University of Georgia 38,837 39 North Dakota State University 16,935
15 Utah State University 37,770 40 University of New Hampshire 16,611
16 University of Missouri 37,074 41 University of Puerto Rico 16,403
17 Colorado State University 34,703 42 University of Alaska Fairbanks 16,370
18 West Virginia University 33,896 43 Montana State University 16,209
19 Virginia Tech 33,164 44 University of Vermont 15,560
20 University of Tennessee 33,096 45 South Dakota State University 14,966
21 Iowa State University 32,981 46 University of Wyoming 14,707
22 Louisiana State University 32,189 47 University of Idaho 14,472
23
University of Massachusetts
(Amherst)
32,000 48 Florida A&M University 14,383
24 Washington State University 30,816 49 New Mexico State University 13,011
25 Oregon State University 29,868 50 University of Maine 12,910
* Data was retrieved from CollegeStats, an informational website, which aggregates publicly available information
provided by the U.S. Department of Education (http://nces.ed.gov) from the 2012-2013 school year

40
Table 3.2: 30 Land-Grant Universities (Simple Random Selection)
01 Ohio State University 15 Utah State University 28
Oklahoma State
University
02 University of Minnesota 16 University of Missouri 29 University of Nebraska
03 University of Florida 17 Colorado State University 31 Auburn University
04 Texas A&M 19 Virginia Tech 32
University of Hawaii
Manoa
05 Michigan State University 20 University of Tennessee 35 Clemson University
08 University of Wisconsin 21 Iowa State University 37
University of Nevada
Reno
11 University of Maryland 22 Louisiana State University 40
University of New
Hampshire
12 University of Arizona 24 Washington State University 42
University of Alaska
Fairbanks
13
North Carolina State
University
25 Oregon State University 47 University of Idaho
14 University of Georgia 26 University of Kentucky 48 Florida A&M
Coding Sheets and Coder Reliability
Coding Sheets
A coding sheet was developed based on the user experience categories and the best
practices associated with each user experience element. The coding sheet was structured as if the
evaluator was a user of the mobile application and therefore began with the visual and sensory
category and continued through the user experience to the interface and navigation design, then
the interaction design and information architecture category, followed by the content and
services offered category, and finished up with the functional and technical specifications
category. The coding sheet examined 49 user experience best practices for mobile applications.
The coding sheet incorporated yes/no, five-point Likert scale, where 5=Strongly Agree,
4=Agree, 3=Neutral, 2=Disagree, and 1=Strongly Disagree, and choose all that apply scoring
options. For all five UX categories there were 34 yes/no scores, 15 scaled scores, and three
questions that utilized select all that apply checkboxes.
Visual Design. There were eight best practices associated with visual design. Three of
the eight best practice items used a yes/no evaluation to indicate whether the best practices

41
associated with contrast, typography, scale, and texture were present or absent. Five of the eight
best practices used a five-point scale to evaluate the degree of consistency in design, the visual
hierarchy, brand identity, overall aesthetics, and use of color (Appendix 1A).
Interface and Navigation Design. There were eleven best practices related to interface
and navigation design that were evaluated. Eight of the eleven best practice items used a Yes/No
evaluation to indicate whether the best practices associated with horizontal scrolling, distortion,
orientation, screen space, button size, back button navigation, labels and icons, and page titles
were present or absent. Two of the eleven best practices used scaled scores to evaluate the degree
of consistency in the navigation and the interface design. The remaining question used
checkboxes to determine which of the following navigation formats were used consistently: (1)
expanding menu, (2) side menu, (3) tabbed menu, and (4) hub and spoke (Appendix 1B).
Interaction and Information Architecture. There were eight best practices related to
interaction design and information architecture that were evaluated. The first best practice –
gestures – incorporated six individual Yes/No evaluation items to determine which gestures the
app employs appropriately. There were five best practices that used a yes/no evaluation to
indicate whether gestures, walkthroughs and coach marks, error messages, input values, and
notifications were present or absent. Two best practices used the five-point scale to evaluate the
degree to which the app encouraged repeated use through interaction and utilized mobile device
features. Finally, one best practice used checkboxes to determine which of the following
information architecture patterns were used: (1) Hierarchy, (2) Hub and Spoke, (3) Nested Doll,
and (4) Tabbed View (Appendix 1C).
Content Strategy: Content and Services Offered. There were eight best practices related
to content and services offered that were evaluated. The first best practice – target audiences –

42
incorporated six individual yes/no evaluation items to determine which university specific
audiences were targeted. Three additional best practice items used a yes/no evaluation to indicate
whether the best practices associated with university name and logo, content sharing, and up-to-
date content were present or absent. Three of the eight best practices used scaled scoring to
determine the degree to which the copy was deemed succinct and non-repetitive, content and
services offered were organized, and multimedia was used appropriately. One of the eight best
practices used checkboxes to indicate which services were offered within the application
(Appendix 1D).
Functional and Technical Specifications. There were nine best practices related to
content and services offered that were evaluated. Eight of the nine best practices used a yes/no
evaluation to indicate whether the best practices associated with load time, privacy policy,
interruptions, continuation, click-to-call, opt-in active consent, user feedback, and functional
progress were present or absent. The remaining best practice used the five-point scale to
determine the effectiveness of the apps’ security and transparency (Appendix 1E).
Coder Reliability
All coding for this study was conducted by the researcher. As an exploratory study
combining quantitative and qualitative approaches a second coder was not employed and thus
inter-coder reliability was not an issue. To ensure consistency in coding the researcher analyzed
each app starting with the visual design and proceeding down the Mobile application user
experience workflow (Figure 2.1) to the functional and technical specifications. Due to the ever-
changing nature of digital media, all applications were downloaded on the same day and the
analysis was performed over a two-week period. The validity of the data measured – the
applications – was high, since there was no opportunity for any of the 30 apps to be updated

43
and/or changed during the analysis time frame. At no point did the researcher go into the app
store and perform any updates to the 30 applications.
Data Collection and Archiving Procedures
To determine the effectiveness of each of the user experience elements and the best
practices associated with each element, the study looked at the home page of the app, the landing
pages for each feature in the main navigation, and all links made available on the landing pages.
This process took the evaluator three levels deep into the app to be sure that there was sufficient
information to evaluate the user experience elements and best practices. If the app was so badly
designed that there was no information to evaluate, a notation was made to indicate that the app
failed in the specified user experience element.
Devices Used
The native applications were evaluated within an iOS platform on an iPhone 5 mobile
device. Since this study was not interested in determining the similarities and differences in
mobile application best practices for different mobile devices, it was ascertained that comparing
the 30 apps on one type of mobile device was a valid approach. The iPhone 5 was selected due to
the availability the researcher had to that particular device.
Pretest
A pretest of the coding sheet was performed for three university apps to validate the
coding sheet and confirm that the order of items within the code sheet was appropriate for coding
expediency. The three universities were randomly selected from the remaining 20 universities of
the original N sample of 50 land-grant universities. The three universities selected were: (1)
Purdue University, (2) South Dakota State University, and (3) University of Connecticut. The
coding sheets were adjusted appropriately based on the pretest results.

44
Visual Records
Screen shots of the application were taken during analysis to provide a visual reference to
the use or lack-thereof, of user experience best practices. Screen shots were saved directly to the
device’s camera and were saved in a dropbox folder following evaluation of the apps. The screen
shots were then used as exhibits and references in the analysis of the data as well as the
discussion of the findings.
Data Analysis Procedures
Quantitative Analysis
After completing the code sheets, scores were generated for each variable in each of the
user experience categories. For the 34 yes/no questions, an answer of Yes was given a 1 and an
answer of No was given a 0. There were 15 five-point Likert scale questions, which were scored
as follows: 5=5 points, 4=4 points, 3=3 points, 2=2 points, and 1=1 point. Checkbox questions
were not included in the calculations, but were used in the in-depth descriptive qualitative
analysis. Once each variable was scored for each mobile application (Appendix 2A-2E),
descriptive statistics were calculated including the frequency of each numbered score for each
variable, as well as the mean and standard deviation for each variable.
From there, the university applications were ranked within each of the five User
Experience categories, in order to determine which universities were the top in each category.
Once the ranking scores for each UX category were determined, a final ranking analysis was
performed to determine which were the top apps across all UX categories.
Qualitative Analysis
Post quantitative analysis, a descriptive qualitative analysis was used to explore the
university apps that were exemplary in the user experience categories. According to Cohen,

45
Manion, and Morrison (2007) descriptive research methods set out to “describe, compare,
contrast, classify, analyze, and interpret the entities and the events that constitute their [the
studies] fields of inquiry” (p. 205). Based on the content analysis and ranking analysis
performed, the six highest-ranking university applications were evaluated in-depth. The
descriptive analysis compared and contrasted the top apps based on the best practice criteria in
each user experience category. The descriptive analysis was supplemented with screen shots
taken during the content analysis. Conclusions about the notable features of the apps were
established through the descriptive qualitative analysis.

46
CHAPTER 4
RESULTS
This chapter describes the findings from the quantitative analysis, addresses the seven
research questions presented in this study, and presents a descriptive qualitative analysis of the
top six mobile applications. The first section answers the first six research questions by
presenting and analyzing the descriptive statistics for each UX category and the ranking of the
mobile apps based on those statistics. The second section answers research question seven and
provides an in-depth narrative about the best practices that were present and absent from the six
overall best applications.
Quantitative Analysis
RQ1: Do university mobile applications achieve functional and technical best practices?
Table 4.1 summarizes the results for the 10 variables that pertain to functional and
technical specifications. (In Tables 4.1-4.5, percentages are provided for dichotomous measures
(yes/no) and means and standard deviations are presented for interval measures in a consolidated
table for easy comparison.
Table 4.1: Descriptive Statistics for Functional and Technical Specifications (Category E)
Variable
Frequency Percentage or
Mean
Std.
Deviation0 1 2 3 4 5
Loads fast, speed is appropriate 8 22 73.3%
Privacy policy provided 25 5 40.0%
Accepts interruptions 0 30 100.0%
Ability to continue task 0 30 100.0%
Click to call capability 4 26 86.7%
Opt in consent 9 21 70.0%
Provides user feedback option 8 22 73.3%
Function progress visually
indicated
5 25 83.3%
Login security used appropriately 1 29 96.7%
Trustworthy and transparent 1 5 9 11 4 3.40 1.04

47
The data reveal that eight of the nine dichotomous best practices had a strong presence
within the 30 mobile applications. The load time and speed of the app were acceptable for the
purpose of the app and did not alter the user experience by being uncontrollable 73% of the time.
All applications were capable of accepting incoming calls while running and effectively resumed
the tasks at the same place or logical restarting point after an interruption (continuation). Click-
to-call, an important functional specification for mobile applications was present 86.7% of the
time. The applications provided some kind of visual indication of activity, such as a progress bar,
83.3% of the time. 73.3% of universities provided the user with an option for providing feedback
about the app, while 96.7% of the apps utilized login security measures in appropriate
circumstances and were not overused. One of the technical best practices for applications is that
the app provides a clearly visible link to the privacy policy and/or terms and conditions.
Providing users with access to information about privacy and security settings is
important, so users have the ability to manage their own privacy and determine whether to use
the app (App Quality Alliance, 2013, p. 26). Only five of the 30 apps achieved this best practice.
There was only one scaled variable evaluated in the study, which dealt with whether the app
provided the user with a strong sense of security and transparency. The App Quality Alliance
(2013) identifies transparency, choice, and control over privacy and security measures as a
guiding principle for app development (p. 26). Overall the applications were rated moderately
trustworthy (M = 3.40, SD = 1.037) suggesting that it was unclear from this study whether the
level of transparency and security practices would create an app environment that users would
feel comfortable in. Overall, the 30 mobile applications seemed to achieve a high level of
functional and technical best practices.

48
RQ2: Do university mobile applications employ best practices related to content and
services offered?
While it is not necessarily a best practice to target all audiences in one mobile
application, it is important to understand whom universities are targeting with their mobile
applications, in order to judge their effectiveness. This section reported on the organization,
timeliness, and breadth of the content and services offered, as well as how easily that content
could be shared with others and whether that content was appropriate for the app. Table 4.2
displays the frequency and percentages or the mean scores and standard deviations for the 13
Content and Services Offered best practices.
Table 4.2: Descriptive Statistics for Content and Services Offered (Category D)
Variable
Mean
Std.
Current students [Audience] 5 25 83.3%
Prospective students [Audience] 24 6 20.0%
Faculty/staff [Audience] 19 11 36.7%
Visitors [Audience] 22 8 26.7%
Donors [Audience] 27 3 10.0%
Alumni [Audience] 24 6 20.0%
Logo and/or name present 7 23 76.7%
Content is easily shareable 18 12 40.0%
Content is up to date 5 25 83.3%
Content is organization 4 11 6 6 3 2.77 1.22
Copy is succinct 2 8 7 9 4 3.17 1.18
Multimedia used appropriately 4 9 5 9 3 2.93 1.26
Offers appropriate services 1 9 14 4 2 2.90 0.92
The frequency statistics reveal that current students clearly were the primary targeted
audience for 83.3% of the applications evaluated. Only 11 universities provided content geared
toward faculty and staff. The remaining university audiences of prospective students (20%),
visitors (26.7%), donors, (10%), and alumni (20%) were less of a focus for the university apps.
Twenty-three of the 30 apps (76.7%) featured the university name and or logo on its welcome or

49
main page. Content proved to be timely and up-to-date in 83.3% of the apps, while only 40%
contained easy access to social media platforms-- with the ability to share content to those
platforms. Statistics for the scaled variables had average scores that were fairly neutral.
Organization of content and services offered (M = 2.77) scored below the midpoint of the scale
and thus could be judged as not overly efficient for easy navigation. It was observed that
organizing menu items alphabetically would have been a simple organizational method that the
apps could have employed. Copy that was reflective of the main points and ideal for quick
information consumption received an average score of M = 3.17. The applications did not
display a great deal of multimedia (M = 2.94) and the services offered were did not consistently
enhance the user experience (M = 2.90). The descriptive statistics divulged that the apps
neutrally adhered to best practices for content and services offered.
RQ3: How are interaction design and information architecture achieved within university
mobile applications?
The descriptive statistics for the variables related to interaction design and information
architecture are represented in Table 4.3.
Table 4.3: Descriptive Statistics for Interaction Design and Information Architecture (Category C)
Variable
Mean
Std. Deviation
0 1 2 3 4 5
Gestures are consistent 0 30 100.0%
Walkthroughs present 24 6 20.0%
Error messages provided 3 27 90.0%
Default input values displayed 4 26 86.7%
Notifications upon exit 14 16 53.3%
Encourages repeated use 6 11 4 3 6 2.73 1.44
Device feature interaction 2 10 2 9 7 3.30 1.34
Consistent architecture pattern 1 5 15 8 1 3.10 0.84
The frequency statistics reveal that three of the five dichotomous variables had a strong
presence within the 30 apps. Appropriate gestures - actions that all major touch operating

50
systems employ that allow users to perform specific tasks (McVicar, 2013) - for the Apple
operating system were employed in all 30 apps. Error messages that were helpful when tasks did
not go as planned were evident in 90% of the apps, while default values for input and appropriate
keyboards for data entry were displayed in 26 out of the 30 mobile apps. When a task takes the
user outside of the mobile application environment and fails to provide notification of that
action, it could be extremely frustrating for the user. Only 16 (53.3%) of the university
applications notified users when an action triggered the use of phone features and other
applications and took the user outside the app environment. Walkthroughs or coachmarks,
which are overlay instructions that show the user how to use different features of the app, were
present in 20% of the apps. With a score of M = 2.73, roughly half of the applications did not
encourage repeated use of the application. Just over half of the applications enabled users to
interact with features of their mobile devices, such as social apps, map applications, and the
camera (M = 3.3). With an overall score of M = 3.10, the use of consistent information
architecture patterns was somewhat inconclusive. The descriptive statistics in Table 4.3
provided great insight into which interaction design and information architecture best practices
the university mobile applications achieved.
RQ4: Which interface and navigation design best practices are prevalent throughout
university mobile applications?
Table 4.4 summarizes the results for the 10 variables that pertain to functional and
technical specifications.
Five of the eight dichotomous variables were prevalent in 80% or more of the 30
applications. Graphics and text were not distorted, blurred, or pixelated in 24 of the applications.
The size of buttons and touch areas were appropriate and page titles were descriptive, concise,

51
and clued the user into the purpose of the page in 25 of the apps. Labels and icons were
descriptive, clear, concise, and consistent within 27 applications and 28 apps supported standard
back button navigation. Of the 30 apps, 60% avoided horizontal scrolling, while 66.7% had an
Table 4.4: Descriptive Statistics for User Interface and Navigation Design (Category B)
Variable
Mean
Std. Deviation
0 1 2 3 4 5
Avoids horizontal scrolling 12 18 60.0%
Clear graphics & text 6 24 80.0%
Supports both orientations 25 5 16.7%
Single screen space (design) 10 20 66.7%
Button size appropriate 5 25 83.3%
Back button functionality 2 28 93.3%
Clear labels & icons 3 27 90.0%
Descriptive page titles 5 25 83.3%
Consistent interface present 13 4 3 3 7 2.57 1.68
Consistent navigation 0 2 4 12 12 4.13 0.90
interface design that was for a full mobile screen, where the main task was front and center.
Over 80% of the apps failed to support landscape orientation on a mobile device. About half of
the apps lacked a consistent interface (M = 2.57), and often directed users to webpages that were
not housed within the mobile application. Buttons and the format of the navigation generally
were consistent (M = 4.13), and the relationship was communicated effectively between
navigation elements and the pages or content they were linked to in 24 of the 30 applications.
The mobile apps complied with user interface and navigation design best practices at a
seemingly high rate.
RQ5: Do university mobile applications utilize visual design best practices?
The frequency, means, and standard deviations for the eight visual design variables are
presented in Table 4.5, revealing how university mobile applications utilized these best practices.
Visual design focuses on the strategic implementation of images, colors, fonts, and other
elements within a mobile app environment (U.S. Department of Health & Human Services,

52
2014). The eight visual design best practices incorporated the concepts of color, contrast,
typography, consistency, visual hierarchy, brand identity, and aesthetics. The use of high contrast
color schemes, or a mix of neutral and bright colors was present in 22 or 73.3% of the apps.
Nineteen apps, 63.3%, used no more than two fonts and those fonts were simple and readable.
Table 4.5: Descriptive Statistics for Visual Design (Category A)
Variable
Mean
Std.
Uses high contrast 8 22 73.3%
Typography simple & readable 11 19 63.3%
Texture & Shape 11 19 63.3%
Consistent & unifying design 6 8 6 4 6 2.87 1.43
Visual hierarchy evident 3 7 8 7 5 3.13 1.25
Consistent brand identity 0 11 7 7 5 3.20 1.13
Strong overall design (Aesthetics) 7 10 4 4 5 2.67 1.42
Color creates structure 1 9 5 9 6 3.33 1.21
Nineteen of the 30 apps also used texture and or shape to enhance detail and make empty
space more appealing. Unity and consistency – where colors, typography, space, and layout
were designed consistently – was lacking throughout many of the apps (M = 2.87), with fourteen
applications earning a one or two for the variable. The visual design of the 30 apps did not
create an overwhelming sense of visual hierarchy (M = 3.13). Twelve apps were deemed to have
strong visual hierarchy, where the elements were arranged according to emphasis in the layout to
create flow, while eight were inconclusive, and 10 lacked a sense of visual hierarchy. The brand
of the university (M = 3.2), or the use of university colors, logos, and university identifiers, was
used consistently throughout only 12 of the university apps. Color was used inconsistently (M =
3.30) to give prominence to content items, demonstrate how items were connected, and provide
structure to content. Aesthetics is essentially the combination of contrast, unity and consistency,
typography, visual hierarchy, brand identity, texture, shape, and space to create a visually

53
pleasing product. The overall beauty or aesthetics of the mobile applications was less than
satisfactory with a score of M = 2.67.
RQ6: Overall, what were the most prevalent and least evident best practices?
To understand where universities performed best and worst in terms of designing
effective apps, the scores for all best practices across the five design levels reported in Tables
4.1- 4.5, were combined into two summary tables. Table 4.6 compiles the frequencies for the 34
dichotomous variables that measured the presence or absence of key features. Table 4.7
summarized the prevalence of best practices for the 15-scaled items based on means. Variables
were listed from highest to lowest, and were coded based on the design category they
represented: A=Visual Design, B=User Interface & Navigation, C=Interaction Design &
Information Architecture, D=Content and Services Offered and E=Technical and Functional
Specifications.
Table 4.6 reveals that among the dichotomous measures only three best practices were
evident in 100% of the apps: gestures, interruptions, and continuation. Meanwhile, four others
were found in at least 90% of the cases: login security, back buttons, labels and icons and error
messages. Overall, it was clear that schools designed their apps primarily for students (evidenced
in 86% of apps) and did not design for multiple or secondary audiences. For example, only 36%
of the apps provided evidence they were intended for faculty, while one-quarter or less appeared
to target visitors, prospective students, alumni or donors. These might be important missed
opportunities.
The areas of poorest performance among the dichotomous variables examined included
the lack of privacy policy disclosures, walkthroughs, and the ability to display content in
landscape orientation. Privacy policy statements are important because they provide users with a

54
sense of transparency and security, which elicits confidence in the app and the organization.
Walkthroughs provide details about using an application, which can be extremely beneficial in
Table 4.6: Overall Prevalence of Best Practices (Dichotomous Variables) Organized by Percent
Variable
Frequency
Percentage
Std.
C-Gestures 0 30 100.0%
E-Interruptions 0 30 100.0%
E-Continuation 0 30 100.0%
E-Login Security 1 29 96.7%
B-Back Button 2 28 93.3%
B-Labels & Icons 3 27 90.0%
C-Error Messages 3 27 90.0%
C-Input 4 26 86.7%
E-Click to Call 4 26 86.7%
E-Function Progress 5 25 83.3%
D-Current [Audience] 5 25 83.3%
D-Up to Date Content 5 25 83.3%
B-Button Size 5 25 83.3%
B-Page Titles 5 25 83.3%
B-Graphics & Text 6 24 80.0%
D-Logo Name 7 23 76.7%
E-Load & Speed 8 22 73.3%
E-User Feedback 8 22 73.3%
A-Contrast 8 22 73.3%
E-Opt in Consent 9 21 70.0%
B-Screen Space 10 20 66.7%
A-Typography 11 19 63.3%
A-Texture & Shape 11 19 63.3%
B-Horizontal Scrolling 12 18 60.0%
C-Notifications 14 16 53.3%
E-Privacy Policy 25 5 40.0%
D-Sharing Content 18 12 40.0%
D-Faculty [Audience] 19 11 36.7%
D-Visitors [Audience] 22 8 26.7%
D-Prospective [Audience] 24 6 20.0%
D-Alumni [Audience] 24 6 20.0%
C-Walkthroughs 24 6 20.0%
B-Orientation 25 5 16.7%
D-Donors [Audience] 27 3 10.0%
{Letter indicates the User Experience Design Category: A = Visual Design, B = User Interface & Navigation
Design, C = Interaction Design & Information Architecture, D = Content & Services Offered, and E = Functional &
Technical Specifications}

55
certain situations and cumbersome in others. In retrospect, it would have been more beneficial to
evaluate walkthroughs based on the effectiveness of including or excluding them from the app.
Landscape orientation is essential in order to play videos, which are being used in increased
numbers to supply information to students as well as others.
Table 4.7 summarizes on a consolidated basis the mean scores and standard deviations
for all 15 scaled items rated by the researcher. Among these, apps performed best on navigation
(M=4.33). As the part of the overall framework for getting around the application, having
consistent buttons and navigational components, that effectively communicate relationships
within the app, is essential. Most of the universities (roughly 24 of 30) succeeded in presenting a
consistent and communicative navigation framework.
Table 4.7: Overall Prevalence of Best Practices (Scaled Variables) Organized by Mean
Variable
Frequency
Mean Std. Deviation
0 1 2 3 4 5
B-Navigation 0 2 4 12 12 4.13 0.90
E-Trustworthy 1 5 9 11 4 3.40 1.04
A-Color 1 9 5 9 6 3.33 1.21
C-Device Features 2 10 2 9 7 3.30 1.34
A-Brand Identity 0 11 7 7 5 3.20 1.13
D-Succinct Copy 2 8 7 9 4 3.17 1.18
A-Visual Hierarchy 3 7 8 7 5 3.13 1.25
C-Architecture Pattern 1 5 15 8 1 3.10 0.84
D-Multimedia 4 9 5 9 3 2.93 1.26
D-Services Offered 1 9 14 4 2 2.90 0.92
A-Unity & Consistency 6 8 6 4 6 2.87 1.43
D-Organization 4 11 6 6 3 2.77 1.22
C-Repeated Use 6 11 4 3 6 2.73 1.44
A-Aesthetics 7 10 4 4 5 2.67 1.42
B-Consistent Interface 13 4 3 3 7 2.57 1.68
Notably the means were below the scale’s midpoint (3.0) for seven measures. The low
performing best practices included multimedia, services offered, unity and consistency,

56
organization and repeated use. The poorest ratings were found for aesthetics and consistent
interface. While aesthetics and consistent interface are best practices associated with two
different UX categories, they are perfect examples of the relationships among the categories and
the best practices. It was evident that an app that had a consistent interface, which directed users
to mobile friendly websites housed within the application, and which fit with the app design
framework, often received a higher score for aesthetics.
Tables 4.6 and 4.7 suggest that there was no distinct pattern for prevalence of best
practices based on UX categories among the 30 applications. Category D, Content and Services
Offered held six of the bottom ten spots for the dichotomous variables. However, it is important
to note, that these variables might have benefited from an alternate evaluation method. It may
have been more telling to evaluate the best practice of target audience as a scaled measure and
simply indicate which audiences the app targeted through a check-box method. No matter the
effectiveness of the audience evaluation factor, there was much room for improvement in the UX
category of Content and Services Offered.
Functional and Technical Specification best practices were among the most prevalent.
Except for the privacy policy best practice, which was present in only 40% of the apps, the
functional and technical specification best practices were existent in at least 70% of the
applications. Overall, visual design (category A) performed poorly among the 30 apps, which no
doubt led to a low mean score for aesthetics (M = 2.67), which takes into consideration all of the
other variables measured. User interface and interaction design (category B), as well as
interaction design and information architecture (category C), had some best practices that the
universities seemed to comply with and others that many of the apps failed to adhere to.

57
Descriptive Qualitative Analysis
User Experience Design was readily examined in Chapter 2, and a model proposed by
Garrett (2011) was used as the conceptual framework for this study (Figure 2.1). The model,
with its five levels (surface, skeleton, structure, scope, and strategy) was used throughout the
study to organize the five user experience categories (Functional and Technical Specifications,
Content and Services Offered, Interaction Design and Information Architecture, User Interface
and Navigation Design, and Visual Design), identify best practices, and develop appropriate
research questions. In order to effectively determine what made the top 6 applications
exemplary, a descriptive qualitative analysis for each of the top 6 applications was performed
based on the model in Figure 2.1.
RQ7: Which university mobile applications were ranked highest based on prevalence of
best practices and what made them exemplary?
Selection Procedure for Identifying Exemplary Apps
In order to determine which universities had the best university mobile applications
overall, a ranking analysis was performed. Five tables were created, one for each user
experience category (Appendix 2A, 2B, 2C, 2D, and 2E) depicting the score each university
received for each of the 49 variables. Anytime an app received the highest possible score for a
category (a 1 for a dichotomous variable or a 5 for a scaled variable) the score was highlighted
The number of highlights each university received was added up to determine which schools
scored best across the 49 best practice variables.
Visual design (Appendix 2A) included 8 variables, so the top score a university could
receive was an 8, if it received all 1s and 5s. User interface and navigation design (Appendix
2B) had 10 variables; interaction design and information architecture (Appendix 2C) contained 8

58
variables, content and services offered (Appendix 2D), was made up of 13 variables, and
functional and technical specifications incorporated 10 variables (Appendix 2E). Once scores for
each UX category were determined for the 30 universities, those scores were added to give each
university an overall ranking score (Table 4.8). Since there were 49 variables or best practices,
the highest score a university could receive was a 49.
The top six schools based on scores were the University of New Hampshire, University
of Arizona, Ohio State University, University of Florida, Washington State University, and
Michigan State University. Their apps were evaluated in a qualitative descriptive analysis to
determine what made these apps exemplary. The University of New Hampshire excelled, with a
high score of 10 out of 10 in User Interface and Navigation Design, a 7 of 8 in Interaction Design
and Information Architecture, and a 12 of 13 for Content and Services Offered. A second place
score of 7, behind Ohio State and University of Florida for Visual Design and a score of 8 for
Functional and Technical Specifications, along with the other three scores were enough to give
the University of New Hampshire the top spot with an overall ranking score of 44. The
University of Arizona, Ohio State University, University of Florida, Washington State
University, and Michigan State University ranked second, third, fourth, fifth, and sixth
respectively. These schools had a range of 6, with highest score being 39 and the lowest being
33. As the next university had a score of 29, making the range between the sixth ranked and
seventh ranked schools a 4, it was determined that a qualitative analysis of the top 6 schools
would be appropriate.

59
Table 4.8: Summary Scores Based on Rankings for University Mobile Applications
University
(A)
Visual
Design
(8)
(B)
User
Interface &
Navigation
(10)
(C)
Interaction Design
& Information
Architecture
(8)
(D)
Content &
Services
Offered
(13)
(E)
Technical &
Functional
Specifications
(10)
Overall
Ranking
Totals
(49)
University of New
Hampshire
7 10 7 12 8 44
University of Arizona 7 10 5 10 6 38
Ohio State University 8 7 7 6 9 37
University of Florida 8 8 4 6 9 35
Washington State
University
7 9 5 6 7 34
Michigan State 5 9 6 5 8 33
Colorado State University 3 9 4 4 9 29
North Carolina State 2 8 5 5 8 28
Oregon State 2 8 6 5 5 26
University of Kentucky 3 6 4 4 8 25
University of Hawaii 3 5 4 4 7 23
University of Nebraska 2 6 4 4 7 23
Auburn University 1 8 4 3 7 23
Florida A&M 3 4 4 3 8 22
Virginia Tech 2 6 3 4 7 22
University of Missouri 3 6 3 2 8 22
Iowa State University 3 6 3 1 9 22
University of Tennessee 3 5 3 3 8 22
Clemson University 2 5 5 3 7 22
University of Minnesota 2 5 5 5 5 22
Texas A&M 3 6 3 2 7 21
University of Nevada 1 5 3 4 8 21
University of Idaho 1 6 3 4 7 21
Oklahoma State 2 7 2 4 5 20
University of Wisconsin 0 3 5 6 6 20
University of Georgia 0 4 3 4 8 19
University of Alaska 1 6 2 4 6 19
Utah State University 1 5 3 5 4 18
University of Maryland 3 6 1 1 6 17
Louisiana State University 0 3 3 3 7 16
{Number indicates the total number of best practice variables measured in each User Experience Design Category}

60
University of New Hampshire
As the top overall ranked school, the University of New Hampshire (UNH) excelled
across all user experience categories. The simplicity, organization, content and services offered
and overall usability of the application made UNH a great app. UNH was one of two schools to
score a perfect 10 out of 10 for its user interface and navigation design. The app successfully
implemented a user interface that was simple and consistent, and navigational components that
were intuitive. The descriptive statistics evaluated in the quantitative analysis revealed that the
majority of applications failed in the best practice areas of orientation and interface consistency.
UNH, along with only four other schools successfully supported portrait and landscape
orientations along with the capability of rapidly transitioning between the two orientations
(Figure 4.1).
The App Quality Alliance (2013) established that it is paramount for developers to create
a mobile application interface that is consistent in terms of action sequences, terms, layouts, and
buttons. It was strongly agreed that UNH had generated an app interface that was indeed
consistent. A majority of the 30 apps directed users to webpages and/or required the user to leave
the application interface, often without informing the user this was happening. UNH created an
interface that never took the user outside of the application framework and that had consistent
sequences, terms, layouts and buttons (Figure 4.2 and 4.3).

61
Figure 4.1: University of New Hampshire portrait and landscape orientations
The organization of the UNH app and its content created a great user interface. The app
utilized two navigation and user interface workflows or action sequences. Figure 4.2 shows how
the app went from the home screen to a secondary screen, in this case Campus Services and
Discover UNH. Both the home and secondary screens utilized labels and icons that were
descriptive, clear, and concise, while informing the user about what content was provided when
items were clicked. The third level consisted of a vertical list of items where each item had a
descriptive title and informative details and an arrow indicating that the item could be clicked on
to get more information. Each screen shot reveals that the icons, labels, button/touch size, and
navigation were appropriate and effective in moving the user through the app.

62
Figure 4.2: University of New Hampshire mobile application interface consistency
Figure 4.3, shows the other action sequence, where the app takes the user to a mobile
website framework that is housed inside the mobile application. UNH utilized this hybrid method
of app development for areas such as dining and events, both of which already had an established

63
mobile web interface that could easily be integrated into the mobile application and still create a
sense of consistency within the app.
Figure 4.3: University of New Hampshire mobile application hybrid mobile web integration
The UNH mobile application utilized a hub and spoke navigation style, where there was a
centralized home screen that allowed users to select a menu option, which took the user to a
specific section that then had its own internal navigation pattern (Figures 4.2 and 4.3). Through
a hub and spoke navigation style, UNH was able to generate a navigation framework that was
consistent and that communicated the relationship between the navigation elements and the
pages and content they were linked to.
The UNH app targeted all audiences, current and prospective students, faculty and staff,
visitors, donors, and alumni, with content that was relevant and that cultivated repeated use.
News, transit, campus maps, campus directory, emergency information, and events were some of
the content items that were seen consistently across the majority of university mobile

64
applications. In addition to these standard services, UNH provided information about the
computer labs on campus, which highlighted the platform (Windows or Mac) and indicated how
many computers were free and in use (Figure 4.4), a feature that was unique to this school.
Figure 4.4: University of New Hampshire computer labs
A dining section was also included in the app that provided hours and information about
each dining center, 10-day menus, and dining news and events. Providing a comprehensive list
of courses offered, with detailed information, the schedule, and instructor details was another
service that other universities either did not have or failed to execute effectively in their own
apps (Figure 4.5).

65
Figure 4.5: University of New Hampshire courses
The University of Arizona
The University of Arizona ranked second overall among the 30 university mobile
applications. The app excelled in the user experience categories of user interface and navigation
design, content and services offered, and visual design. Like the UNH app, the Arizona app
received a perfect score, a 12 out of 12, for its user interface and navigation design. The icons
and labels were communicative of the content provided when touched and the labels were
aesthetically pleasing in their design and simplicity (Figure 4.6). The Arizona app used high
contrast color with white text on red and black text on white. The brand identity of Arizona was
evident throughout the app with the use of these high contrasting color schemes and the
prominent university logo and name on the home screen. The consistency of the interface and
navigation was one of the apps’ overall strengths. The second- and third- level pages continually
used shape and color to differentiate between headings and different content items. Arrows

66
indicated that a section was in fact a button and that there was further information when clicked
(Figure 4.6). The consistent navigational components as well as the strong visual hierarchy
allowed users to move back and forth throughout the app with ease. Arizona, like UNH, used a
hybrid approach to mobile app development with the use of mobile websites housed within the
mobile application.
Figure 4.6: University of Arizona user interface and visual design
Most notable was how well the various mobile websites fit into the overall design of the
app (Figure 4.7). Arizona was one of the only schools to address accessibility issues by allowing
users to change the size of the text (Figure 4.8). The app also utilized breadcrumbs, further
enabling the user to know where they had been and where they were within the mobile app
framework. Another notable feature of the Arizona app was the successful targeting of various
audiences. Users had the ability to select whether they were a current student, a prospective
student, or alumni, and the services of the app would be filtered to match the selected audience
category (Figure 4.8). The Arizona app offered an extensive number of services, many of which
would encourage repeated use of the application. Events, news, contacts, athletics, a campus
map, emergency information, and public transportation were some of the standard services

67
present. In addition, the Arizona app provided access to the bookstore, campus health, campus
recreation, and library mobile websites. Dining center, meal plan, alumni, and donor campaign
information were also available. Multimedia was used appropriately in the sections dedicated to
videos, tours, and social media. The app enabled users to share content through social media and
bookmark content within the app. The user had the ability to save events to the mobile device
calendar, as well as open directional information in the device’s map application (Figure 4.9).
The University of Arizona mobile application demonstrated that all of the user experience
categories affect the others and that a successful app applies best practices from each category.
Figure 4.7: University of Arizona mobile websites within mobile app

68
Figure 4.8: University of Arizona font size and audience options
Figure 4.9: University of Arizona share, bookmark, calendar, and view in map options
Ohio State University
Visual appeal and design seemed to be the top priority for the Ohio State University
(OSU) mobile application, while less emphasis was placed on the interface, navigation,

69
interaction, and architectural components, as well as the content and services offered. The overall
design of the app was aesthetically and sensually pleasing. The use of color, shape, and contrast,
were used to create unity and an effective visual hierarchy within the app. The Ohio State brand
was apparent throughout the app through the use of the famous Ohio State “O” and the Buckeye
colors of red, white, and grey. The Ohio State homepage was one of the best designed among all
the apps. The use of dark and light grey and white to create sections on the homepage was
consistent with current trends in web and mobile design. Icons and labels effectively portrayed
the services that the OSU app had to offer and were visually appealing (Figure 4.10).
Despite top marks in visual design, the app fell short with its user interface, navigation
design, and content and services offered. That is not to say that the university failed completely
in these areas, but it was apparent that less emphasis was placed on the importance of these user
experience categories. The app did not support landscape orientation and while it successfully
utilized a hybrid framework, the websites housed within the app were not mobile, but desktop
versions that required the user to zoom in and scroll horizontally to see all content. While the app
did offer standard university services such as a campus map, news, events, bus information, a
directory, course information, and athletics, the execution of these services was substandard.
Instead of providing a list of courses for the user to browse through, searching was the only
option available. The athletics section provided a brief summary of the news with text that was
small and difficult to read and then took the user to the athletics desktop website for the full
article.

70
Figure 4.10: Ohio State University application home page. This figure illustrates screen shots that depict the
continuity of the continuous scroll capabilities of the OSU homepage and the additional options in the “Near your
current location” section of the homepage.
The app did encourage repeated use, especially with its homepage features. News, events,
and continuously updated photos would most likely cultivate daily interaction with the app. Most
notable on the homepage was the “near your current location” portion. Here visitors to the app
could find the nearest parking, shopping, and food to their current location. The parking feature

71
even included information about the availability of spaces and the access capabilities of each
parking facility (Figure 4.10). Ohio State’s distinction in visual design and functional and
technical specifications along with its mediocre interface and navigation design, interaction and
information architecture, and content and services offered resulted in an overall third place
ranking.
University of Florida
The biggest strength of the University of Florida (UF) mobile application was the
simplicity of the design and navigation. The home page provided the UF logo and name, clearly
indicating the brand identity of the app, and had a vertical list of services separated by lines, with
arrows that indicated the option to move forward and get more information. The app consistently
used the UF colors of blue, orange, and white and used white and black text to provide contrast
between background colors and text. Icons were well designed and reflective of the services and
content offered within each section (Figure 4.11).
Figure 4.11: University of Florida home screen. This figure illustrates the continuous scroll feature of the home
screen depicted in three separate screen shots.

72
Navigation throughout the app was fluid, offering forward and back options consistently
in the top left corner of the app, however, the tabbed menu navigation that appeared at the
bottom of the screens within the various sections, was somewhat difficult to use due to the small
size of the touch areas. The app’s interface was consistent by keeping the user within the app and
directing users to mobile websites rather than desktop sites. However, the different mobile
websites detracted from the overall visual appeal of the app. The library section utilized a
different blue that clashed with the app’s color scheme and had icons that were pixelated, while
the rec sports menu used yet another shade of blue, as well as black header bars (Figure 4.12).
Sending the user to mobile websites located within the app framework with distinctly different
navigation configurations was confusing. It was difficult to distinguish between the mobile
application and mobile website navigation. For instance, the home back button of the app was
often clicked, when the user wanted to go back within the mobile website architecture.
Figure 4.12: University of Florida mobile websites

73
The content of the UF app was comprehensive, covering emergency information, maps,
transportation, e-learning, class schedules, libraries, dining, events, a directory, athletics,
recreation sports, news, social media, photos, and video, and even offered an option to login and
make a payment to the user’s university account. There was an option to provide feedback about
the app, links to other UF apps and social media accounts and information about the app and the
University of Florida. One great feature of the app was the ability to customize the home screen
by showing or hiding the various modules or services (Figure 4.13). Overall, the app had great
services to offer, but the execution of those services was not entirely effective. The interface,
visual, and navigational components were good but fell short of excellence.
Figure 4.13: University of Florida customization screens
Washington State University
The Washington State University mobile app had strong visual appeal with its use of
compelling photos within the app. As an app with its primary focus on prospective students, the
photo backgrounds on the home page, the directory, about WSU, and the gallery provided a great

74
glimpse of life at WSU (Figure 4.14). Although photos as backgrounds are often a distraction
from content, the app was successful in using an effect, where the photo was blurred when a user
scrolled down through the content (Figure 4.15) to make the content more readable.
Figure 4.14: Washington State University photography
The use of color and shape created an effective visual hierarchy throughout the app. Most
notable were the use of white header bars with grey and red text or red bars with white text to
indicate what page the user was on and the red highlighted menu item also denoting the current
page (Figure 4.16). The About WSU page’s use of color blocks to separate the sections of
content added to the app’s attractiveness. The location detail pages were well designed while
also providing relevant content and options to view the location in the map or get directions
(Figure 4.16). Icons were standard, simple designs that would be recognized by most mobile
users. Navigation buttons and format were consistent and effectively communicated the
relationship between navigation elements and the pages or content they were linked to and page
titles helped to strengthen this relationship.

75
Figure 4.15: Washington State University blurred photo effect
The app used two distinct navigation patterns, side menu and tabbed menu. The main
navigation was housed in the side menu while the tabbed menu was present on the home and
map pages to provide additional options to the user (Figure 4.16). The app had one of the most
consistent interfaces and never directed users to webpages, mobile or desktop. Roughly half of
the mobile applications provided an indicator or notification when an action would take the user
outside of the application framework. WSU was one of the apps that provided notifications
across multiple situations. Users were prompted prior to making a phone call and were notified
when leaving the app to access directions. Despite the visual appeal and strong interface and
navigational components, the WSU app lacked content that would encourage repeated use of the
app. The services that the WSU app offered were indicative of an app designed to target
prospective and possibly current students. The about page, gallery, and map services provided a
great deal of information that would be of benefit to students and parents considering the

76
university, but would hardly elicit repeated use of the app by these audiences. The alerts, news,
events, and tweet sections might also be informative for prospective students, but could also be
beneficial for current students, interested in getting involved or wanting to know what’s
happening on campus. While these were great services, the app lacked some of the offerings
other universities had that would most likely entice students to use the app on a daily basis. The
WSU app would benefit from further exploration into what content their users would like to see.
Figure 4.16: Washington State University navigation and visual hierarchy
Michigan State University
The Michigan State app branded as the Spartan App, was one of the most intriguing, and
perhaps the one app that would be used on a regular basis. The focus of the app was about
getting current students, faculty and staff engaged on and off campus. The app offered deals,
food options, event information, nightlife options, transportation, and maps. What made this app
so different from the other 29 was the collaboration between MSU and the surrounding East

77
Lansing community. Businesses were able to contact MSU and advertise opportunities within the
college app (Figure 4.17).
Figure 4.17: Michigan State University app home, advertising, and navigation screens. The navigation screenshot
depicts a sliding side menu, where the content is pushed right to reveal the navigation, when the three white line
icon is pressed.
The places section contained information, directions and phone numbers for banks, car
shops, cinemas, dentists, dry cleaners, gas stations, grocery stores, fitness, laundry, liquor stores,
salons/barbers, storage, tailors, and tanning, many of the everyday necessities of a college
student (Figure 4.18). The food section had information about MSU dining, and locations for
various types of cuisine including, American, Asian, breakfast, coffee, dessert, Indian, pizza or
pasta, Mexican, sandwiches, Mediterranean, and an option that informed the user which
restaurants were currently open (Figure 4.19). The transportation services included Capital Area
Transportation Authority or public transportation, taxi, and shuttle information. A campus news
feed area provided the latest information about athletic events and deals around town. The main

78
content areas that were notably absent from the MSU app were emergency information and
academic related services.
Figure 4.18: Michigan State University places section. These screen shots represent three levels of information
within the places section of the app.
Figure 4.19: Michigan State University food section. These screen shots represent three levels of information within
the food section of the app.

79
The app was very well designed and the interface allowed for easy navigation and use of
the app. A side menu housed the main navigation and a tabbed menu was used on various pages
to provide further navigation options. The information architecture pattern was a nested doll,
where the user was able to move forward and back through the content and click the side menu
icon to access the other sections whenever desirable. Labels and icons were descriptive, well
designed, and consistent throughout the app and did a great job of informing the user what
content to expect. The use of photos as a design component, as is evident in the food and deals
sections was sensually and visually appealing. Brand colors of green, white, and grey, high
contrast color schemes, and the use of shape created a powerful visual hierarchy and an app that
had great consistency and unity.

80
CHAPTER 5
DISCUSSION
Summary
The general conclusion from both the quantitative and qualitative analysis was that none
of the apps excelled in all five UX categories, indicating that it may in fact be difficult to be
exemplary in all five of the categories and comply with the best practices of the Mobile
application user experience framework (Figure 2.1). The top six apps seemed to focus on two or
three of the UX categories rather than all five, resulting in stronger user experiences overall. The
highest ranked six apps portrayed how the various UX categories and best practices interact with
and rely on each other to create a successful user experience. The in-depth analysis revealed that
it was very difficult to look at the first three levels of an app once and effectively evaluate the
user experience best practices. In fact, on further analysis of the top six ranked apps, where 30
minutes to an hour was spent going through the app and writing an evaluation of the UX best
practices, different ratings might have been obtained. While the top six apps could have flip-
flopped positions, the researcher believed that these six apps were far better in most of the UX
categories than the other 24 apps.
Visual Design with User Interface and Navigation Design
The UX categories of visual appeal and user interface and navigation design were the two
that had the most impact on one another. An interface that was designed well had strong visual
appeal and those apps that were visually appealing tended to have effective navigation that
enhanced the visual design and user interface. The apps that integrated these two categories most
effectively were the University of Arizona, Washington State University, and Michigan State

81
University. The other three apps, University of New Hampshire, University of Florida, and Ohio
State University, while effective apps took users to either mobile websites or desktop websites
within the mobile app framework, but this hybrid approach to app design detracted from the
overall design and appeal of the site and often led to confusion about navigation.
Content and Services Offered with Interaction Design and Information Architecture
Content and services offered and interaction design and information architecture were the
other two UX categories that were closely related. The content and services offered were
indicative of whether the app encouraged repeated use. It was the use of information architecture
patterns that determined the organization of the content or services offered within the app. The
app that offered the best content, a strong information architecture, encouraged repeated use, and
enabled interaction with mobile device features most effectively was the University of Arizona.
The University of New Hampshire offered a variety of unique content and was successful in
targeting a variety of university audiences. The University of Florida allowed users to filter
content and customize the home screen of the app. Michigan State did a great job of connecting
students to the East Lansing community through its content, but failed to integrate more campus
related features within the app. Washington State University’s content was not very conducive to
repeated use, but was more informational for prospective students. All six applications would
have benefited from better organization of the services offered, perhaps by alphabetizing the
service options on the home page or in the main navigation menu. Another option would have
been to group the services into content categories, though this may have been difficult to
achieve. University of New Hampshire had the best organization of content by offering sections
such as academics, campus services, and discover UNH, which each had further options within.

82
Functional and Technical Specifications
This UX category had the least variance among all 30 apps and was therefore not
critiqued to the same extent as the other four categories. The mere fact that this category had so
many yes/no questions, made it less intriguing, as each feature was either present or absent from
the apps. Aside from the privacy policy best practice, all other functional and technical
specifications were predominately present in nearly all of the mobile applications. The
University of Florida, Ohio State University, and Washington State University did not provide
users access to their privacy policy, while the University of Arizona was slow to load pages,
which affected the user experience significantly. Overall the top six apps performed well in the
best practices associated with the UX category of functional and technical specifications.
The Impact of Application Source on Best Practices
After the quantitative and qualitative analysis were performed, the evaluator determined
whether the apps were (1) custom apps designed at the universities, “write it yourself options” or
(2) white label apps, “free, out of the box options” (Dixon, 2012). During the content analysis
portion of this study notations were made about what development option each university had
chosen for their app. If the university used a white label app, the vendor was determined where
possible. Table 5.1 summarized whether the app was a custom or white label and the developer
or vendor respectively for each university application.

83
Table 5.1: App Type and Developer or Vendor
University Custom White Label Developer/Vendor
University of New Hampshire X UNH Mobile
University of Arizona X Modo Labs
Ohio State University X Unknown
University of Florida X UF Information Tech
Washington State University X WSU Mobile App Team
Michigan State X X Guidesmob & Graphic Designer AnMaree
Colorado State University X Ellucian
North Carolina State X NC State Students and Staff
Oregon State X Guidebook
University of Kentucky X Blackboard
University of Hawaii - - Unknown
University of Nebraska X Blackboard
Auburn University X AU Information Tech
Florida A&M X Blackboard
Virginia Tech X Unknown
University of Missouri X Mizzou Information Tech
Iowa State University X College of Engineering & Information Tech
University of Tennessee X Blackboard
Clemson University X Unknown
University of Minnesota X Guidebook
Texas A&M X Blackboard
University of Nevada X DUBLabs
University of Idaho X Ellucian
Oklahoma State X OSU Students & Office of Communications
University of Wisconsin X UW Communications & Information Tech
University of Georgia X UGA Students & Information Tech
University of Alaska X Blackboard
Utah State University X Unknown
University of Maryland X EX Axess
Louisiana State University X Blackboard

84
Fifteen of the 30 universities developed their own custom apps, while fourteen, used
various vendors’ white label apps. Only one application, the University of Hawaii, provided no
indication of how it was developed. The University of New Hampshire, Ohio State University,
University of Florida, and Washington State University all developed custom mobile
applications with a development team in-house at the university. Michigan State University used
a white label app from a company called Guidesmob, but customized the app with the help of a
graphic designer. The University of Arizona used a custom white label app called Kurogo, which
is an open-source program created by Modo Labs (2015). The University of Arizona no doubt
customized their white label app to conform to university brand standards and was able to take a
pre-designed application framework and develop an app that was both visually appealing, had a
strong user interface, and great content. While the University of Arizona was the only land-grant
university evaluated in this study to use the Kurogo app framework, other universities across the
nation using the same software, include Boston College, Colgate, Dartmouth, Harvard, College
of William & Mary, Georgetown, Indiana State, and New York University. The mix of custom
and white label apps among the 30 ranked schools provides no indication as to which method is
better. Planning and execution of UX best practices supersede the type of app development any
one university chose.
Limitations
There were four principal limitations in this study. The first limitation involves the
exploratory nature of the study itself. No previous research could be located that explored how
universities approach mobile communication and the development of mobile applications. Thus,
this study was the first of its kind, and the criteria to evaluate the university apps was drawn from
best practices associated with mobile websites and general mobile applications, as opposed to

85
university mobile applications. Based on the research, 49 reasonable best practices were chosen
as criteria to evaluate the 30 applications, but there could be additional variables or best practices
to consider when evaluating university mobile applications. In particular, certain definitions of
the variables, such as walkthroughs, could be clarified and redefined. The topics explored in the
study are in their infancy with regard to measurer research methodology.
The second limitation deals with the sample size of the study. A sample size of 30
universities was very small-scale and limited the reliability and validity of the study, particularly
the quantitative results. However, as a mixed-methods study that incorporated quantitative
content and qualitative descriptive analyses, evaluating 30 applications was deemed reasonable.
However, increasing the sample size would no doubt increase the statistical power of the study.
A more in-depth quantitative analysis might have been possible with a larger sample size and
might have provided more insight into the relationships between individual best practices and
UX categories.
Third, the validity of the study also was potentially limited by the use of one coder for the
content analysis. The use of one coder could have led to bias, based on the evaluator’s
preconceived perceptions and opinions of what a good app should look like and how it should
work. Adding a second coder could improve the validity and reliability of the findings if the
study is replicated in the future.
Using only land-grant universities was a fourth limitation of this study. While it was
reasonable to use 30 of the 115 land-grant schools in light of Colorado State’s role as Colorado’s
only land-grant university, using different samples of the more than 3,000 colleges and
universities in the United States could have produced different patterns and conclusions. Despite
the use of only land-grant universities, this study can be used a guide for not only other land-

86
grant schools, but for other aspiring universities as well. Evaluating non land-grant universities
in a similar fashion would only add power, validity, and reliability to the overarching concepts of
university mobile applications, best practices, and user experience design.
Future Research
More research needs to be done with mobile applications in general as well as about
mobile applications and their role in university communications with students and other
constituents.
Validating best practices measures
Specifically for universities, a re-evaluation of the best practice criteria and application of
the same study to a larger sampling of universities would be highly beneficial to substantiate the
results and provide further insight into university mobile applications and user experience best
practices. Performing studies on each of the user experience levels and or categories would
provide universities with a greater understanding of each UX category by itself and how the five
interact and relate to one another to create the overall user experience of a mobile application. In
particular, it would be valuable to develop an empirically testable set of measures that could be
reliably combined into effective indices for each of the five dimensions or user experience design
categories investigated in this study. Achieving reliability would involve creating a set of
interval-level measures (scaled items) for which a Cronbach Alpha statistic could be created.
Understanding mobile app strategies
Looking back at the Mobile Application User Experience Workflow presented in Chapter
2 (Figure 2.1), the user experience strategy level was excluded from this particular study because
of the difficulty of verifying the strategy behind the observable content. Garrett (2011) noted
that, “The foundation of successful user experience is a clearly articulated strategy” (p. 35). As

87
this study was concerned with user experience design and therefore the four levels of scope,
structure, skeleton, and surface, there was no definitive way to determine what the university
mobile app strategy was, nor user needs or expectations. Therefore, it would be of great
advantage to perform a variety of studies that address university goals as well as user needs,
desires, and interests.
Qualitative and quantitative investigations involving university officials and app
developers could be used to determine not only the strategy of the university with regard to app
development, but also to corroborate the conclusions found in this study. Whether the research is
focus groups, surveys, questionnaires, or in-depth interviews, all universities would benefit from
knowing what strategies are currently employed and considered effective, as well as how
developers see the future of mobile in the university environment.
Investigating user response
Beyond strategy involving research with professionals, at the opposite end of the mobile
application user experience workflow is the user. Performing quantitative and qualitative studies
to determine what students, faculty, staff, visitors, alumni, donors, and any other university
audiences want from a mobile application could help determine the future of mobile
communication at universities. Usability testing would be another approach for getting feedback
about how well a university or group of universities is doing from a user experience perspective,
such research can assist in determining where a developer needs to go next. In fact, research has
already begun that investigates the users’ perspective of university mobile applications. Gupta,
Gop, and Kyei-Blankson (2014), published a study in October that evaluated the usage patterns
and student expectations of two popular university owned mobile applications. “The study

88
highlights the gaps where universities fail to reach out to its students, hence delaying their
engagement or use of the apps” (Gupta, Gop & Kyei-Blankson, 2014, p. 742).
Mobile communication research is in its infancy, but no doubt has a bright future as more
and more people purchase multiple types of devices. Universities can use this research to
evaluate how well their own apps conform to UX best practices and the leading land-grant
university mobile applications. Those that wish to further the research in the field of mobile
communication in universities can build upon this study as a framework for future investigations
surrounding user experience and university mobile applications.

89
REFERENCES
Al-Khalifa, H. S. (2014). A framework for evaluating university mobile websites. Online
Information Review, 38, 166-185. http://dx.doi.org/10.1108/OIR-12-2012-0231.
Amini, M., Blair, M. A., Forrester, J., Goldstein, S. J., Katsouros, M., Rocchio, R. A., . . .
Williams, A., III. (2013). Developing a campus mobile strategy: Guidelines, tools, and
best practices. Retrieved from http://www.educause.edu/library/resources/developing-
campus-mobile-strategy-guidelines-tools-and-best-practices.
App Quality Alliance. (2013). Best practice guidelines for producing high quality mobile
applications. Retrieved from
http://www.appqualityalliance.org/files/AQuA_best_practices_doc_v2_3_final_june_201
3.pdf.
App Quality Alliance. (2014). AQuA and Membership. Retrieved from
http://www.appqualityalliance.org/aqua-and-membership.
Association of Public and Land-Grant Universities. (2012). The land-grant tradition. New York,
NY: Association of Public Land-Grant Universities. Retrieved from
http://www.aplu.org/library/the-land-grant-tradition/file.
Barnes, S. B. (2010). Visual evaluation of the World Wide Web. In S. Josephson, S. B. Barnes,
& M. Lipton (Eds.), Visualizing the web: Evaluating online design from a visual
communication perspective (165-182). New York, NY: Peter Lang Publishing Inc.
Bennett, J. G. (2005). Design fundamentals for new media. Clifton Park, NY: Thomson Delmar
Learning.

90
Best Practices. (2012). In S. D. Hill (Ed.), Encyclopedia of Management (7th ed., pp. 56-57).
Detroit: Gale. Retrieved from
https://csuglobal.idm.oclc.org.ezproxy2.library.colostate.edu/login?url=http://go.galegrou
p.com.ezproxy2.library.colostate.edu:2048/ps/i.do?id=GALE%7CCX4016600030&v=2.
1&u=coloradosu&it=r&p=GVRL&sw=w&asid=dcce60d8f637e2503c924769624bc10d
Bishop, T. (2012). Mobile in higher education (infographic). Retrieved from
http://icamp.us/blog/.
Blackboard Mobile. (2011). Mobile strategy: Let’s create a better mobile experience, together.
Retrieved from https://www.blackboard.com/platforms/mobile/services/blackboard-
mobile-strategy.aspx.
Bowen, K., & Pistilli, M. D. (2012). Student preferences for mobile app usage. Retrieved from
http://www.educause.edu/library/resources/student-preferences-mobile-app-usage.
Bretschneider, S., Marc-Aurele Jr., F. J. (2005). “Best practices” research: A methodological
guide for the perplexed. Journal of Public Administration Research and Theory, 15(2),
307-323.
Budiu, R. (2013, Sept. 14). Mobile: Native apps, web apps, and hybrid apps. Retrieved from
http://www.nngroup.com/articles/mobile-native-apps/.
Burton, V. L. (2011). Best practices. In Encyclopedia of small business (4th
ed). (pp. 115-117).
City, ST: Gale, Cengage Learning 115-117.
Busch, C., De Maret, P. S., Flynn, T., Kellum, R., Le, S., Meyers, B., . . . Palmquist, M. (1994-
2012). Content analysis: Writing@CSU. Colorado State University. Retrieved from
http://writing.colostate.edu/guides/guide.cfm?guideid=61.

91
Cardello, J. (2014). The difference between information architecture (IA) and navigation.
Nielsen Norman Group. Retrieved from http://www.nngroup.com/articles/ia-vs-
navigation/.
Cerejo, L. (2012). Guidelines and best practices: The elements of the mobile user experience.
Smashing Magazine. Retrieved from
http://www.smashingmagazine.com/201/07/12/elements-mobile-user-experience.
Christ, A. M. (2011). Bridging the mobile app gap. Sigma: Inside the digital ecosystem, 27-32.
Cohen, L., Manion, L. & Morrison, K. (2007). Research methods in education. New York, NY:
Routledge.
Dahlstrom, E. & diFilipo, S. (2013). BYOD and consumerization of IT in higher education
research. EDUCAUSE. Retrieved from http://www.educause.edu/library/resources/byod-
and-consumerization-it-higher-education-research-2013.
Design. (n.d.). In Wikipedia. Retrieved October 30, 2014, from
http://en.wikipedia.org/wiki/User_experience_design.
DigitalGov. (2014). Mobile user experience guidelines and recommendations. Retrieved from
http://www.digitalgov.gov/resources/mobile-user-experience-guidelines-and-
recommendations/.
Dixon, C. (2012, November 1). Finding the right one: Mobile technology in higher education.
EDUCAUSE Review. Retrieved from http://www.educause.edu/ero/article/finding-right-
one-mobile-technology-higher-education.
EDUCAUSE (2014). About Educause. Retrieved from http://www.educause.edu/about.
Fling, B. (2009). Mobile Design and Development. Sebastopol, CA: O’Reilly Media, Inc.
Galitz, W. O. (2007). The essential guide to user interface design: An introduction to GUI design

92
principles and techniques. Indianapolis, IN: Wiley Publishing, Inc.
Garrett, J. J. (2010). The elements of user experience: User-centered design for the web and
beyond. Indianapolis, IN: New Riders Publishing.
Gupta, P., Gop, K. & Kyei-Blankson, L. (2014). College students’ usage of and expectations
from university owned mobile applications. In Proceedings of World Conference on E-
Learning in Corporate, Government, Healthcare, and Higher Education 2014 (pp. 742-
745). Chesapeake, VA: Association for the Advancement of Computing in Education
(AACE).
Harvey, A. (2013). User experience: What is it and why should I care? Usability Geek. Retrieved
from http://usabilitygeek.com/user-experience/.
Heitkotter, H., Hanschke, S., Majchrzak, T. A. (2012). Evaluating cross-platform development
approaches for mobile applications. Web Information Systems and Technologies. 8th
International Conference, 140, 120-138.
Hellbom, K. (2013). Mobile web apps as an alternative to native mobile apps: The future of
mobile web apps on the competitive marketplace (Degree thesis).
International Ergonomics Association. (2014). Definition and domains of ergonomics. Retrieved
from http://www.iea.cc/whats/index.html.
ISO DIS 941-210:2010. Ergonomics of human system interaction – Part 210: Human-centered
design for interactive systems. International Standardization Organization (ISO).
Switzerland.
Jobe, W. (2013). Native apps vs. mobile web apps. International Journal of Interactive Mobile
Technologies, 7(4), 27-32.

93
Johnson, D., Means, T., & Khey, D. N. (2013). A state of flux: Results of a mobile device survey
at the University of Florida. EDUCAUSE. Retrieved from
http://www.educause.edu/ero/article/state-flux-results-mobile-device-survey-university-
florida.
Jones, C., Ramanau, R., Cross, S., & Healing, G. (2010). Net generation or digital natives: Is
there a distinct new generation entering university? Computers & Education, 54(3), 722–
732.
Klein, D. (2012). How to decide: Mobile websites vs. mobile apps. Adobe Website Retrieved
from http://www.adobe.com/inspire/2012/02/mobile-websites-vs-mobile-apps.html.
Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E, & Sinnelä, A. (2011). UX
curve: A method for evaluating long-term user experience [Abstract]. Interacting with
Computers: The Interdisciplinary Journal of Human-Computer Interaction, 23, 473-483.
Leggett, D. (2011). Considerations for mobile design (part 1): Speed. UX Booth. Retrieved rom
http://www.uxbooth.com/articles/considerations-for-mobile-design-part-1-speed.
Lew, P. & Olsina, L. (2013). Towards understanding and improving mobile user experience.
Excerpt from Pacific NW Software Quality Conference 2013 Proceedings. Retrieved
from http://www.uploads.pnsqc.org/2013/papers/t-011_Lew_paper.pdf.
Madden, M., Lenhart, A., Maeve, D., Sandra, C., & Gasser, U. (2013). Teens and Technology
2013. Pew Research Center, Washington: DC. Retrieved from
http://www.pewinternet.org/2013/03/13/teens-and-technology-2013/
McVicar, E. (2012). Designing for mobile part 1: Information architecture. UX Booth. Retrieved
from http://www.uxbooth.com/articles/designing-for-mobile-part-1-information-
architecture/.

94
McVicar, E. (2013). Designing for mobile part 2: Interaction design. UX Booth. Retrieved from
http://www.uxbooth.com/articles/designing-mobile-part-2-interaction-design.
McVicar, E. (2014). Designing for mobile part 3: Visual design. UX Booth. Retrieved from
http://www.uxbooth.com/articles/designing-mobile-part-3-visual-design/.
McWherter, J. & Gowell, S. (2012). Professional mobile application development. Indianapolis,
IN: John Wiley & Sons, Inc.
Mikkonen, T., & Taivalsaari, A. (2011, October). Apps vs. open web: The battle of the decade.
In Proceedings of the 2nd Workshop on Software Engineering for Mobile Application
Development (pp. 22-26).
Modo Labs (2015). About Modo. Retrieved from https://www.modolabs.com/about-modo/
Morville, P. (2004). User Experience Design. Semantic Studios. Retrieved from
http://semanticstudios.com/user_experience_design/.
Nielsen, J. (2011, October 10). Mobile content: If in doubt, leave it out. Nielsen Norman Group
Website Retrieved from http://www.nngroup.com/articles/condense-mobile-content/.
Nielsen, J. & Norman, D. (2014). The definition of user experience. Nielsen Norman Group
Website Retrieved from http://www.nngroup.com/articles/definition-user-experience/.
Nielsen Company (2014, February 10) The digital consumer. Retrieved from
http://www.nielsen.com/us/en/reports/2014/the-us-digital-consumer-report.html.
Park, J., Han, S. H., Kim, H. K., Oh, S., & Moon, H. (2013). Modeling user experience. A case
study on a mobile device. International Journal of Industrial Ergonomics, 43, 187-196.
Psomas, S. (2007). The five competencies of user experience design. UX Matters. Retrieved
from http://www.uxmatters.com/mt/archives/2007/11/the-five-competencies-of-user-
experience-design.php.

95
Rainie, L. (2013, June 6). Cell phone ownership hits 91% of adults. Pew Research Center.
Retrieved from http://www.pewresearch.org/fact-tank/2013/06/06/cell-phone-ownership-
hits-91-of-adults/.
Ritter, N. L. (2012). Mobile technologies use report: Department of Student Life Studies. Texas
A&M University
http://studentlifestudies.tamu.edu/sites/studentlifestudies.tamu.edu/files/results/full/171-
full.pdf.
Sambasivan, D., John, N., Udayakumar, S., & Gupta, R. (2011). Generic framework for mobile
application development. Internet (AH-ICI) Second Asian Himalayas International
Conference, Kathmundu, Nepal (pp. 1-5).
Serrano, N., Hernantes, J., & Gallardo, G. (2013). Mobile Web Apps. Software Technology:
IEEE Computer Society, 30(05), pp. 22-27.
http://doi.ieeecomputersociety.org/10.1109/MS.2013.111.
Simon, H. A. (1963). The sciences of the artificial. Cambridge, MA: MIT Press.
Stempel, G. H. III (2003). Content analysis. Mass communication research and theory. In G. H.
Stempel, D. H. Weaver, & G. C. Wilhoit. (Eds.), Mass communication research and
theory (pp. 209-219). Boston, MA: Pearson Education, Inc.
Stenstrom, M. L. & Laine, K. (2006). Searching for principles of good practices in practice-
oriented assessment. In Stenstrom, M. L. & Laine, K. (Eds.), Towards good practices for
practice-oriented assessment in European vocational education (pp. 11-19). Jyväskylä,
Finland: University of Jyväskylä, Institute for Educational Research.

96
Stern, C. (2014). CUBI: A user experience model for project success. UX Magazine, (1314).
Retrieved from http://uxmag.com/articles/cubi-a-user-experience-model-for-project-
success.
Traeg, P. (2014, February 11). Four ways to build a mobile application, part 3: PhoneGap.
Smashing Magazine. Retrieved from
http://mobile.smashingmagazine.com/2014/02/11/four-ways-to-build-a-mobile-app-
part3-phonegap/.
Trochim, W.M. (2006). The Research Methods Knowledge Base, 2nd Edition. Web Center for
Social Research Methods. Retrieved from http://www.socialresearchmethods.net/kb/.
United States Department of Agriculture. (2014). National Institute of Food and Agriculture:
NIFA land-grant colleges and universities. Retrieved from
http://www.ams.usda.gov/AMSv1.0/getfile?dDocName=STELPRDC5107924.
U.S. Department of Health & Human Services. (2014). User experience basics. Retrieved from
http://www.usability.gov/what-and-why/user-experience.html.
Vesely, A. (2011). Theory and methodology of best practice research: A critical review of the
current state. Central European Journal of Public Policy, 5(2), 98-117.
Wimmer, R. D. & Dominick, J. R. (2006). Mass media research: An introduction. Belmont, CA:
Thomson Wadsworth.
World Wide Web Consortium. (2008). Mobile web best practices 1.0: Basic guidelines.
Retrieved from http://www.w3.org/TR/mobile-bp/.
World Wide Web Consortium (2014) About W3C. World Wide Web Consortium Website.
Retrieved from www.w3.org/Consortium.

97
APPENDIX 1A
VISUAL DESIGN CODE SHEET
UNIVERSITY_________________________________________________________________
BEST PRACTICE SCORING NOTES
A - Contrast
App uses high
contrast color
schemes (mix of
neutral and bright
colors)
YES
NO
AX - Unity &
Consistency
Colors, typography,
space, and layout are
designed consistently
and create a sense of
unity throughout the
app
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
A - Typography
Fonts and text
elements are simple
and readable; No
more than 2 fonts are
used throughout the
app
YES
NO
AX - Visual
Hierarchy
App creates a sense
of visual hierarchy
and flow (i.e.
elements are
arranged according
to emphasis in the
layout – through the
use of different font
sizes, colors, spacing
elements, and content
sections)
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree

98
AX - Brand Identity
The brand of the
university is
consistent throughout
the app with the use
of university colors,
logos, and/or brand
identifiers (i.e.
slogans)
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
A - Texture &
Shape
App uses texture
and/or shape to
enhance detail and
make empty space
more appealing
YES
NO
AX - Aesthetics
The overall design of
images, colors, fonts,
and other elements
are visually and
sensually pleasing;
design enhances user
engagement and
interest
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
AX - Color
Color is used to give
prominence to
content items,
demonstrate how
items are connected,
and helps structure
content (i.e.
separates content)
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree

99
APPENDIX 1B
INTERFACE AND NAVIGATION DESIGN CODE SHEET
UNIVERSITY_________________________________________________________________
B - Horizontal Scrolling
App avoids horizontal
scrolling
YES
NO
B - Graphics/Text
Graphics and text are not
distorted, blurred or
pixelated
YES
NO
B - Orientation
App supports landscape and
portrait orientation and is
capable of rapid transition
between orientations
YES
NO
B - Screen Space
Design is for a single
window or full screen.
Main task is front and
center
YES
NO
BX - Consistent Interface
Design
The interface of the app is
consistent – when app
directs users to webpages
they are mobile friendly and
fit with the app interface
design
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree

100
B – Button Size
App uses appropriately
sized buttons and touch
targets (i.e. buttons are easy
to click and react
appropriately to touches)
YES
NO
BX - Navigation
Buttons and format of
navigation are consistent;
navigation communicates
the relationship between
navigation elements and the
pages or content they are
linked to
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
B - Back Button
Navigation
App supports/uses standard
back button navigation
YES
NO
B - Labels & Icons
Labels and Icons are
descriptive, clear, concise,
and consistent (inform the
user about what content will
be provided when clicked)
YES
NO
B - Page Titles
App provides descriptive yet
concise page titles that clue
the user into the content of
the page
YES
NO
Navigation Styles
Which navigation format(s)
is/are used? Select all that
apply
EXPANDING MENU: A menu icon (generally a series of lines) is contained in the
header. Tapping the icon shows a list of main menu items.
SIDE MENU: You will often press an icon with a series of lines or slide the screen over,
this then expands or slides open the menu, showing a list of menu options vertically
down the side of the screen. This can have categories as well as being scrollable.
TABBED MENU: A persistent toolbar, displayed in either the header or footer of an
app, allows user to quickly switch between sections.
HUB & SPOKE: A centralized home screen allows the user to select a menu option. This
will navigate the user to this section, which can have its own internal navigational
pattern. The user may return to the home screen by activating a back link.

101
APPENDIX 1C
INTERACTION DESIGN AND INFORMATION ARCHITECTURE CODE SHEET
UNIVERSITY_________________________________________________________________
C - Gestures
App employs gestures that
are consistent with the
operating systems standards
YES
NO
C - Walkthroughs
Overlay instructions or
coach marks are provided
upon initial launch to show
users how to use different
features of the app when
deemed necessary
YES
NO
C - Error Messages
App provides helpful error
messages when a task does
not go as planned (i.e.
messages when login fails)
YES
NO
C - Input
Default values are displayed
for input and appropriate
keyboard is available when
entering data (i.e. numbers
are available when asking for
a phone number)
YES
NO
CX - Repeated Use
App encourages repeated use
through interaction with
features of the app
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree

102
CX - Device Features
App enables interaction with
mobile device features
(camera, location functions,
and other apps available on
the device)
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
C - Notifications
App notifies user when it is
leaving the app to access
other phone features and/or
apps
YES
NO
CX - Architecture Pattern
App utilizes a consistent
architecture pattern that is
shallow not deep in nature
(i.e. hierarchy, hub & spoke,
tabbed view, nested doll)
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
Identify which of the
following information
architecture patterns the
app uses
HIERARCHY: Standard index and
series of sub pages
HUB & SPOKE: Central index/hub users
navigate out to “spokes” and back to “hub”
NESTED DOLL: Linear pattern – leads
to more detailed info.
Perception of moving forward & back
TABBED VIEW: Group of sections
linked together through a toolbar or menu

103
APPENDIX 1D
CONTENT AND SERVICES OFFERED CODE SHEET
UNIVERSITY_________________________________________________________________
D - Target Audiences
Which of the following
audiences does the app
target? Select all that
apply.
Current Students
Yes No
Prospective Students
Yes No
Faculty/Staff
Yes No
Campus Visitors
Yes No
Donors
Yes No
Alumni
Yes No
DX - Organization
Content and services
offered are organized
efficiently for easy
navigation (i.e. by target
audience)
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
DX – Succinct Copy
Copy is reflective of the
main points that are ideal
for quick information
consumption. Additional
info is deferred to
secondary screens
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
D - University
Logo/Name
The university name
and/or logo appear on the
landing page or main
page of the app
YES
NO

104
DX - Multimedia
Images and video are
used appropriately to
supplement content and
provide a good user
experience
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
D - Sharing Content
App has easy access to
social media platforms
and the ability to share
content to these platforms
YES
NO
D - Up-to-Date Content
Content is updated with
current and accurate
information
YES
NO
DX - Services Offered
App offers services that
are appropriate for a
university mobile app
environment; services
offered are usable and
enhance the user
experience
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
Which services are
available on the mobile
application?
(Taken from Al-Khalifa,
2014; Marshall, 2012;
and Hayes & Walker,
2013)
Academic
Programs
Admission
Information
Alumni Athletics
Bookstore
Bus/Shuttle
Information
Campus
Map
Campus Tour
Class
Schedule
Contact
Course
Catalog
Dining Info
Directory
of
people/staff
Emergency Events Grades
Library Multimedia Social media

105
APPENDIX 1E
FUNCTIONAL AND TECHNICAL SPECIFICATIONS CODE SHEET
UNIVERSITY________________________________________________________________
E - Load time & Speed
Load time & speed of app
is acceptable for the
purpose of the app and
does not alter the user
experience by being
uncontrollable
YES
NO
EX - Trustworthy
App provides user with a
sense of security and is
transparent (informs users
about what personal
information is required,
who it will be shared with,
and why)
1 Strongly Disagree
2 Disagree
3 Neutral
4 Agree
5 Strongly Agree
E - Privacy Policy
App provides a clearly
visible link to the privacy
policy and/or terms and
conditions
YES
NO
E - Interruptions
App has the capability to
accept an incoming call
while running
YES
NO
E - Continuation
App has the capability to
resume tasks from the same
place or a logical
restarting point
YES
NO

106
E - Click-to-Call
App provides phone
numbers that can be
clicked and prompt the
user to make a call
YES
NO
E - Opt-In Consent
App obtains opt-in consent
prior to accessing location
data and/or geo-tagging
with photos and videos;
provides an option to opt-
out
YES
NO
E - User Feedback
App provides users with a
means to contact the
organization with
questions and/or provide
feedback about the app
YES
NO
E - Function Progress
App provides visual
indication of activity (i.e.
prompt for user input,
splash screens, progress
bars, “please wait” text,
etc.)
YES
NO
E - Login Security
App asks users to login
appropriately (i.e. when
accessing personal info).
Logins are not overused
YES
NO

107
APPENDIX 2A
RANKINGS FOR VISUAL DESIGN
UNIVERSITY
A
CONTRAST
AX
UNITY &
CONSISTENCY
A
TYPOGRAPHY
AX
VISUAL
HIERARCHY
AX
BRAND
IDENTITY
A
TEXTURE
& SHAPE
AX
AESTHETICS
AX
COLOR
A
TOTAL
University of Florida 1 5 1 5 5 1 5 5 8
Ohio State University 1 5 1 5 5 1 5 5 8
University of Arizona 1 5 1 5 5 1 4 5 7
Washington State University 1 5 0 5 5 1 5 5 7
University of New Hampshire 1 5 1 5 4 1 5 5 7
Michigan State 1 3 1 4 4 1 5 5 5
Florida A&M 1 1 1 1 2 1 2 3 3
Texas A&M 1 1 1 2 2 1 2 2 3
University of Kentucky 1 3 1 4 4 1 4 4 3
University of Hawaii 1 5 1 4 2 0 2 4 3
University of Missouri 1 3 1 3 3 1 2 3 3
Iowa State University 1 3 1 4 2 1 3 4 3
University of Tennessee 1 1 1 2 2 1 2 2 3
Colorado State University 1 2 1 2 5 0 1 2 3
University of Maryland 1 2 1 3 2 1 1 4 3
Oregon State 1 4 1 4 4 0 4 3 2
Oklahoma State 1 2 1 3 2 0 1 2 2
Virginia Tech 0 2 1 3 3 1 2 2 2
North Carolina State 1 4 1 3 3 0 3 4 2
University of Nebraska 1 2 1 3 3 0 1 2 2
Clemson University 1 2 0 4 2 0 2 4 2
University of Minnesota 1 3 0 3 4 1 3 4 2
University of Nevada 0 4 0 4 4 1 4 4 1
University of Alaska 0 2 0 2 2 1 2 2 1
University of Idaho 0 4 0 3 4 1 3 4 1
Auburn University 1 2 0 2 2 0 2 3 1
Utah State University 0 1 0 1 2 1 2 2 1
University of Georgia 0 1 0 1 3 0 1 2 0
Louisiana State University 0 1 0 2 3 0 1 3 0
University of Wisconsin 0 3 0 2 3 0 1 1 0

108
APPENDIX 2B
RANKINGS FOR INTERFACE AND NAVIGATION DESIGN
UNIVERSITY
B
HORIZONTAL
SCROLLING
B
GRAPHICS/
TEXT
B
ORIENTATION
B
SCREEN
SPACE
BX
CONSISTENT
INTERFACE
B
BUTTON
SIZE
BX
NAVIGATION
B
BACK
BUTTON
B
LABELS &
ICONS
B
PAGE
TITLES
B
TOTALS
University of Arizona 1 1 1 1 5 1 5 1 1 1 10
University of New Hampshire 1 1 1 1 5 1 5 1 1 1 10
Michigan State 1 1 0 1 5 1 5 1 1 1 9
Washington State University 1 1 0 1 5 1 5 1 1 1 9
Colorado State University 1 1 1 1 4 1 5 1 1 1 9
Oregon State 1 1 0 1 5 1 5 1 0 1 8
North Carolina State 1 1 0 1 2 1 5 1 1 1 8
University of Florida 1 1 0 1 5 0 5 1 1 1 8
Auburn University 1 1 1 1 1 1 3 1 1 1 8
Oklahoma State 1 1 0 1 1 1 4 1 1 1 7
Ohio State University 0 1 0 0 5 1 5 1 1 1 7
Virginia Tech 1 0 0 1 2 1 4 1 1 1 6
Texas A&M 1 1 0 1 1 1 4 1 1 0 6
University of Kentucky 0 1 0 1 3 1 4 1 1 1 6
University of Alaska 1 1 0 1 1 1 4 1 1 0 6
University of Idaho 0 0 1 0 2 1 5 1 1 1 6
University of Missouri 1 0 0 1 1 1 3 1 1 1 6
University of Nebraska 1 1 0 1 1 0 3 1 1 1 6
Iowa State University 0 1 0 0 3 1 5 1 1 1 6
University of Maryland 1 1 0 1 1 1 4 1 0 1 6
University of Nevada 0 0 0 1 4 1 4 1 1 1 5
University of Hawaii 0 0 0 0 4 1 5 1 1 1 5
University of Tennessee 1 1 0 1 1 0 3 1 1 0 5
Utah State University 1 1 0 1 2 1 4 0 0 1 5
Clemson University 0 1 0 0 1 1 4 1 1 1 5
University of Minnesota 0 1 0 0 3 1 4 1 1 1 5
Florida A&M 0 1 0 0 1 1 4 1 1 0 4
University of Georgia 0 0 0 0 1 1 4 1 1 1 4
Louisiana State University 0 1 0 0 1 0 2 1 1 0 3
University of Wisconsin 0 1 0 0 1 0 2 0 1 1 3

109
APPENDIX 2C
RANKINGS FOR INTERACTION DESIGN AND INFORMATION ARCHITECTURE
UNIVERSITY
C
GESTURES
C
WALKTHROUGHS
C
ERROR
MESSAGES
C
INPUT
CX
REPEATED USE
CX
DEVICE
FEATURES
C
NOTIFICATIONS
CX
ARCHITECTURE
PATTERNS
C
TOTALS
Ohio State University 1 1 1 1 5 5 1 2 7
University of New Hampshire 1 1 1 1 5 5 1 3 7
Oregon State 1 1 1 1 5 2 1 1 6
Michigan State 1 0 1 1 5 5 1 3 6
North Carolina State 1 0 1 1 3 5 1 2 5
University of Arizona 1 0 1 1 5 5 0 4 5
Washington State University 1 0 1 1 2 2 1 5 5
University of Wisconsin 1 0 1 1 4 5 1 2 5
Clemson University 1 1 1 1 2 4 1 2 5
University of Minnesota 1 1 1 1 3 2 1 4 5
Florida A&M 1 0 1 1 2 3 1 4 4
University of Kentucky 1 0 1 1 2 2 1 3 4
University of Hawaii 1 0 1 1 2 2 1 4 4
University of Nebraska 1 0 1 1 2 4 1 3 4
University of Florida 1 0 1 1 5 2 0 4 4
Auburn University 1 0 1 1 3 4 1 3 4
Colorado State University 1 1 1 0 1 5 0 4 4
Virginia Tech 1 0 1 1 2 3 0 4 3
Texas A&M 1 0 1 1 2 4 0 3 3
University of Georgia 1 0 1 1 2 2 0 2 3
University of Nevada 1 0 1 0 4 4 1 3 3
University of Idaho 1 0 1 1 3 2 0 3 3
University of Missouri 1 0 1 0 4 4 1 4 3
Louisiana State University 1 0 1 1 1 4 0 3 3
Iowa State University 1 0 1 1 1 2 0 3 3
University of Tennessee 1 0 1 1 2 1 0 3 3
Utah State University 1 0 1 1 1 4 0 3 3
Oklahoma State 1 0 0 1 1 2 0 3 2
University of Alaska 1 0 0 1 2 4 0 3 2
University of Maryland 1 0 0 0 1 1 0 3 1

110
APPENDIX 2D
RANKINGS FOR CONTENT AND SERVICES OFFERED
UNIVERSITY D
CURRENT
D
PROSPECTIVE
D
FACULTY
D
VISITORS
D
DONORS
D
ALUMNI
DX
ORGANIZATION
DX
SUCCINCT
COPY
D
LOGO/NAME
DX
MULTIMEDIA
D
SHARING
CONTENT
D
UPTODATE
CONTENT
DX
SERVICES
DTOTALS
University of New Hampshire 1 1 1 1 1 1 5 5 1 5 1 1 4 12
University of Arizona 1 1 0 0 0 1 5 5 1 5 1 1 5 10
Washington State University 1 0 1 1 0 0 4 4 1 5 0 1 3 6
University of Wisconsin 1 0 1 0 1 1 2 3 1 2 0 1 3 6
University of Florida 1 0 0 0 0 0 2 5 1 3 1 1 5 6
Ohio State University 1 0 0 1 0 0 3 5 1 4 1 1 3 6
Oregon State 0 1 0 1 0 0 2 4 1 4 1 1 3 5
Michigan State 1 0 1 1 0 0 3 4 0 4 1 1 2 5
North Carolina State 1 0 1 0 0 0 4 4 1 3 1 1 3 5
Utah State University 1 0 1 0 1 1 1 2 1 2 0 0 4 5
University of Minnesota 0 1 0 1 0 0 4 4 1 2 1 1 3 5
Oklahoma State 1 0 0 0 0 1 3 2 1 2 0 1 1 4
Virginia Tech 1 0 1 0 0 0 4 3 1 2 0 1 3 4
University of Kentucky 1 0 0 0 0 1 3 4 1 4 0 1 2 4
University of Georgia 1 0 1 0 0 0 2 2 1 2 0 1 3 4
University of Nevada 1 0 0 1 0 0 3 4 0 4 0 1 3 4
University of Hawaii 0 1 0 1 0 0 5 2 0 4 0 1 3 4
University of Alaska 1 1 0 0 0 0 2 2 1 4 0 1 2 4
University of Idaho 1 0 1 0 0 0 4 2 1 1 0 1 4 4
University of Nebraska 1 0 0 0 0 0 2 3 1 3 1 1 2 4
Colorado State University 1 0 0 0 0 0 1 3 1 1 1 1 3 4
Florida A&M 1 0 0 0 0 0 2 2 1 3 0 1 2 3
Louisiana State University 1 0 0 0 0 0 1 1 1 3 1 0 2 3
Auburn University 1 0 0 0 0 0 2 4 0 1 1 1 3 3
University of Tennessee 1 0 1 0 0 0 2 3 1 2 0 0 2 3
Clemson University 1 0 1 0 0 0 4 2 0 2 0 1 4 3
Texas A&M 1 0 0 0 0 0 2 3 1 4 0 0 2 2
University of Missouri 1 0 0 0 0 0 2 4 0 4 0 1 3 2
Iowa State University 0 0 0 0 0 0 3 3 0 2 0 1 3 1
University of Maryland 0 0 0 0 0 0 1 1 1 1 0 0 2 1

111
APPENDIX 2E
RANKINGS FOR FUNCTIONAL AND TECHNICAL SPECIFICATIONS
UNIVERSITY
E
LOAD&SPEED
EX
TRUSTWORTHY
E
PRIVACY
POLICY
E
INTERRUPTION
E
CONTINUATION
E
CLICKTOCALL
E
OPTIN
CONSENT
E
USER
FEEDBACK
E
FUNCTION
PROGRESS
E
LOGIN
SECURITY
E
TOTALS
University of Florida 1 5 0 1 1 1 1 1 1 1 9
Iowa State University 1 3 1 1 1 1 1 1 1 1 9
Ohio State University 1 5 0 1 1 1 1 1 1 1 9
Colorado State University 1 5 1 1 1 1 0 1 1 1 9
Florida A&M 1 4 0 1 1 1 1 1 1 1 8
Michigan State 1 3 1 1 1 1 0 1 1 1 8
North Carolina State 1 4 0 1 1 1 1 1 1 1 8
University of Kentucky 1 3 0 1 1 1 1 1 1 1 8
University of Georgia 1 3 0 1 1 1 1 1 1 1 8
University of Nevada 1 4 0 1 1 1 1 1 1 1 8
University of Missouri 1 3 0 1 1 1 1 1 1 1 8
University of Tennessee 1 3 1 1 1 1 1 1 1 0 8
University of New Hampshire 1 4 1 1 1 1 0 1 1 1 8
Virginia Tech 1 4 0 1 1 1 1 0 1 1 7
Texas A&M 0 4 0 1 1 1 1 1 1 1 7
Washington State University 1 3 0 1 1 1 0 1 1 1 7
University of Hawaii 1 2 0 1 1 1 1 0 1 1 7
University of Idaho 1 5 0 1 1 0 1 1 0 1 7
Louisiana State University 0 4 0 1 1 1 1 1 1 1 7
University of Nebraska 0 4 0 1 1 1 1 1 1 1 7
Auburn University 1 4 0 1 1 1 1 0 1 1 7
Clemson University 1 3 0 1 1 1 0 1 1 1 7
University of Arizona 0 3 0 1 1 1 0 1 1 1 6
University of Alaska 0 2 0 1 1 1 1 1 0 1 6
University of Wisconsin 0 4 0 1 1 1 1 1 0 1 6
University of Maryland 0 1 0 1 1 1 1 0 1 1 6
Oregon State 1 4 0 1 1 0 0 0 1 1 5
Oklahoma State 1 2 0 1 1 1 0 0 0 1 5
University of Minnesota 1 2 0 1 1 0 0 0 1 1 5

Valoris-Thesis

More Related Content

Similar to Valoris-Thesis (20)

Valoris-Thesis