Planning, Marketing and Assessing a Digital Library:The Open University Digital Archive (OUDA)

The Open University Digital Archive (OUDA)
Phase 1:
I. OUDA literature review
II. Planning OUDA as a preservation repository
III. Marketing OUDA preservation capabilities to internal stakeholders
IV. Assessing OUDA as an accessible preservation system
Stephen J. Stose
Metadata Officer
The Open University
Library Services

Table of Contents

I. OUDA literature review (p. 3)
Introduction and literature review scope
Overview
Planning OUDA
Digital preservation planning and procedure
Preservation as business strategy
Repository infrastructure
Marketing OUDA
Marketing and communications strategy
Stakeholder involvement
Profiling for feedback and evaluation
Assessing OUDA
The OUDA evaluation framework: The triptych model
Performance
Usefulness
Usability
Conclusions
II. Planning OUDA as a preservation repository (p. 19)
Introduction
Relationship to library strategic planning
Purpose
The strategic context
Relevant Literature
Stakeholder and user needs analysis
STELLAR Project: Preliminary stakeholder views assessed
OU Library stakeholders
University stakeholders

2

Students and public stakeholders
Stakeholder benefits, costs and risks
General benefits
Costs and risks
Demand and impact
Recommendations for action
The goals and outcomes of OUDA
Responsible parties
Action plan and timeline
Preliminary communications plan
Forecast budget for OUDA
Pilot testing and scalability
SWOT analysis
Limitations and assumptions
Reflections
III. Marketing OUDA preservation capabilities to internal stakeholders (p. 41)
Introduction
Relevant literature
Marketing goals and outcomes
Positioning statement
Target audience and key messages
Target audience
Key messages
Message delivery strategies
Tools, timeline, and responsible parties
Marketing costs
Mockups of selected marketing methods
Posters
Informational posters and handouts
Project postcard / info-graphic
Reflection
IV. Assessing OUDA as an accessible preservation system (p. 57)

Introduction

Assessment plan for outcomes (with relevant Literature)
Assessment rubric
Outcome 1: Policy plan in place
Outcome 2: Trusted Repository Audit (TRAC)
Outcome 3: Digital object prototypes established
Outcome 4: Repository interface established (and is usable)
Outcome 5: Metadata is linkable
Outcome 6: Stakeholder feedback on usefulness (STELLAR re-visited)
Reflection
Bibliography

3

Phase 1:
OUDA literature review
Introduction and literature review scope
This paper presents a review of some of the important contributions to the literature on
planning, marketing, and assessing a developmental prototype of a digital repository to
archive and preserve university generated content. Its plan for development is in response
to increasing demand from Open University (OU) stakeholders to provision content
management services for university-generated material that requires long-term sustainable
access and preservation management. The Open University Digital Archive (OUDA) is our
solution to providing core information and enterprise content management systems to
service the needs of the Open University’s online educational model and corporate e-
business.
The literature review forms part of a discussion document. It relates to the initial phase of
the development of the digital archive (i.e., OUDA) per se. More specifically, it will form the
basis towards the discussion of three important aspect of developing this digital archive
project:
1) Planning OUDA as a preservation repository
2) Marketing OUDA preservation capabilities to internal stakeholders
3) Assessing OUDA as an accessible preservation system
The planning documentation will discuss the business model project plan for OUDA and
how it is to satisfy both library and university strategic planning. Marketing the prototype
will focus specifically on our plan to position OUDA with regards to its target audience and
stakeholders (e.g., administrators, researchers, lecturers, staff, and to a lesser extent
students). A public and student-focused release belongs to a later phase. The assessment
will specify a plan to test the system landscape and whether it can appropriately satisfy its
stated service outcomes as an accessible system of preservation.
This is not an exhaustive literature review. For a nearly exhaustive bibliography of recent
papers related to institutional repositories1
and digital curation and preservation2
, please
see the work of Charles W. Bailey (2011, 2010, respectively).

1
http://www.digital-scholarship.org/irb/irb.html
2
http://www.digital-scholarship.org/dcpb/dcpb.htm

4

Overview
We will focus here on two overarching goals deemed critical to the successful
implementation of a digital archive system to hold and preserve the university’s legacy
material. They are:
A. Preservation policy and repository infrastructure
B. Access (usefulness and usability) and content re-use
Broadly envisioned, these goals will also form the basis for the global outcomes upon
which our evaluations and assessment of Phase One planning will hang. These global
outcomes are rooted firmly in the OU Library’s long-term mission and strategic priorities,
which are nested within and conform to university priorities. These are specified more
expansively in the planning documentation below.
Briefly, as library for this online university, our chief mission is to achieve efficiency savings
for faculty and researchers who develop content. For this reason, the current paper will
review best practices and standards with regards to achieving the long-term preservation
of legacy course material within a robust, extensible, and modular systems architecture
that we propose to build.
Our chief goal for Phase One planning (a two-year plan) is encompassed in the statement
below:
Good preservation infrastructure and planning will facilitate the long-term usable
and useful access to materials previously used (i.e., legacy) in course e-production
and presentation. This will facilitate the re-use of non-current OU learning material,
enabling it to serve as a model for future course material production and reduce its
unnecessary duplication, and serve to contextualize the history and enhance the
reputation of OU’s model of higher online education.
Planning OUDA
Digital preservation planning and procedure
The Research Libraries Group (2002) defines digital preservation somewhat differently
than most readers are accustomed to thinking about physical preservation. They write:
Digital preservation is defined as the managed activities necessary: 1) For the long
term maintenance of a byte stream (including metadata) sufficient to reproduce a
suitable facsimile of the original document and 2) For the continued accessibility of
the document contents through time and changing technology.
An excellent starting point for planning a digital repository for the digital preservation is the
book by Ross Harvey, Digital Curation: A How-to-do-it Manual (2010), and its focus on
organization, staffing, and costs, as well as its adherence to the Digital Curation Centre’s
(DCC) lifecycle model3
, itself an indispensible resource. Complementing this is Alex Ball’s
(2010) DCC report Preservation and Curation in Institutional Repositories. There is also an

3
http://www.dcc.ac.uk/resources/curation-lifecycle-model

5

online tutorial in Digital Preservation Management4
sponsored by Cornell University
Library.
Additionally, every year the Digital Preservation Training Programme5
(DPTP) is held at
the University of London Computer Centre. The Digital Preservation Coalition6
, a non-profit
organization dedicated to the advice and advocacy of enduring digital collections,
sponsors the event.
These essential resources include sections on repository software, preservation
architectures and planning tools, metadata and its tools, media obsolescence, as well as
digital object identifier schemes. They also place fundamental stress on conforming to the
Open Archival Information System’s (OAIS) reference model, published by the
Consultative Committee for Space Data Systems (2102), and reproduced in Figure 1
below.

Figure
1

The OAIS model focuses on the long-term impact of changing technologies, multimedia
and data formats, and a changing user community (Consultative Committee for Space
Data Systems, 2002). Lavoie (2004) and the Digital Preservation Coalition have published
an introductory guide to this model, which explains the concept of an information package.
The information package contains three basic elements required to ensure that digital
materials can be preserved independent of specific technology, and one that guarantees a
degree of object persistence (see LeFurgy, 2002, for a good explanation). They are:
1) Content information (the bits/bytes with details on bit interpretation)
2) Preservation information (provenance, fixity/authenticity, reference)
3) Descriptive information (identification metadata)
This single logical package is usually structured in an XML metadata wrapper (i.e.,
FOXML, METS). The submitted information package (SIP), delivered by its producer to be
archived, may or may represent the same form when the package is stored as an archived
information package (AIP). Nor will it necessarily represent the same form when the
package is accessed as a dissemination information package (DIP). That is, when users
actually see how the archived object is presented and described on-screen.

4
http://www.dpworkshop.org/dpm-eng/contents.html
5
http://www.dptp.org/
6
http://www.dpconline.org/

6

Subsequent efforts made by the Working Group on Digital Archive Attributes (OCLC-RLG,
2002) expanded on the OAIS model by outlining concrete attributes and responsibilities of
a repository for it to be deemed trustworthy. It is known as the Trustworthy Digital
Repository checklist (TDR), also known as ISO 16363. With the goal of developing a
certification framework with prescriptive intent, the RLG-NARA Digital Repository
Certification Task Force (2007) elaborated on the TDR list of attributes to develop the
Trustworthy Repositories Audit and Certification (TRAC7
) standard. TRAC now serves as
an authoritative audit that serves as a checklist of the necessary evidence required to
demonstrate long-term repository viability.
Many advocate for making repository software more “preservation-aware,” and some
(Kaczmarek et al., 2006) have suggested incorporating the TRAC checklist into their
evaluation of repository software. Others have studied the direct effect these tools have on
their repositories. For instance, Vardigan and Cole (2007) applied the OAIS model to a
social science archive. Cornel University applied TRAC to a data staging repository to
study trust in preservation chains (Steinhart, Dietrich & Green, 2009). HathiTrust is also
committed to TRAC as its primary assessment tool, and its website outlines how the
Center For Research Libraries (CRL), which now administers TRAC, has successfully
audited the HathiTrust repository8
.
TRAC is not the only preservation checklist (see TDR, above), but will be the one applied
to OUDA during Phase 1 implementation. Others include:
• Data Asset Framework (DAF)9
• Digital Repository Audit Method Based on Risk Assessment (DRAMBORA)10
• The Open Planets Foundation (OPF)11
• Life Cycle Information for E-Literature (LIFE)12
Preservation as business strategy
Any preservation policy, however, cannot be developed in isolation, but must be linked to
core institutional business strategy. LIFE, for instance, is a methodology developed by
University College London and the British Library that allows an organization to model the
digital lifecycle of a repository project and determine preservation costs over the next 5, 10
and 20 years.
Some authors (Becker, Kulovits, Guttenbrunner, et al., 2009) make a pragmatic distinction
between concrete preservation plans and high-level policies that regulate and respond to
institutional strategy. For instance, the ICPSR Digital Preservation Policy Framework13
outlines the high-level factors important to establishing an organization’s commitment to
the digital repository. The Northeast Document Conservation Center14
similarly aids in
preservation planning at a higher conceptual level.

7
http://www.crl.edu/archiving-preservation/digital-archives/metrics-assessing-and-certifying-0
8
http://www.hathitrust.org/trac
9
http://www.dcc.ac.uk/resources/repository-audit-and-assessment/data-asset-framework
10
http://www.dcc.ac.uk/resources/repository-audit-and-assessment/drambora
11
http://www.openplanetsfoundation.org/
12
http://www.life.ac.uk/
13
http://www.icpsr.umich.edu/icpsrweb/content/datamanagement/preservation/policies/dpp-framework.html
14
http://www.nedcc.org/assets/media/documents/DigitalPreservationSelfAssessmentfinal.pdf

7

A JISC commissioned study (Beagrie, Semple, Williams et al., 2008) provides a good
model to follow when formulating high-level digital preservation policy. They specifically
outline the priorities digital repository managers must address, if the preservation
repository is to survive over the long-term as part of the organizational business culture
(see also Seamus, 2012). For the current review, I combine the priorities of this JISC
study with the priorities the ICPSR Digital Preservation Policy Framework have outlined.
These are:
• Organizational viability: How will preservation serve organizational need, and who
are its stakeholders?
• How will it be integrated into and how does it relate to other organizational strategic
priorities?
• What are the objectives of preservation and how will these be supported?
• Has the content it will potentially store been defined and delimited? What kinds of
collections will it store?
• Who is procedurally accountable to these policies and what are their obligations?
• Who is financially and organizationally responsible for sustaining the repository?
While these high-level policy guides provide important and useful recommendations by
setting a framework for concrete planning, they do not provide actionable and concrete
steps towards ensuring long-term access to the repository. For this reason we, like Becker,
Kulovits, Guttenbrunner, et al. (2009), think it potentially useful to distinguish between a
set of high-level preservation policies and a preservation plan. Similarly, the JISC
commissioned study (Beagrie, Semple, Williams et al., 2008) also distinguishes between
high-level “policy clauses” and concrete “implementation clauses.”
In any event, most institutions do not differentiate, but include both high-level and concrete
implementation actions under the rubric of “preservation policy.” That is, the state of
preservation policy around the globe is still quite developmental. Sheldon (2013), as a
Fellow in the Library of Congress, analyzed 33 digital preservation policies, strategies and
plans (variously named) in order to develop a taxonomy of topics covered by the
documents. What is useful about this is that she includes links to the 33 policies
themselves, equally divided between Europe and North America, and libraries and
archives.
In their “Survey of Institutional Readiness,” Kenney and Buckley (2005) found only about
half of 114 organizations with digital repositories had preservation policies written, while a
mere third had vetted and implemented these across management levels. While slightly
broader than the topics Sheldon extracted, they cover the same kind of questions
librarians and archivists need to ask themselves when planning a repository. They include:
• Costs and long-term funding arrangements for maintaining the repository
• Staffing roles and responsibilities (organizational and technical expertise)
• Submission guidelines (who can ingest what, and the criteria for its selection)
• Descriptions of the collection and the objects it will contain
• Object format guidelines
• Authentication mechanisms (to track users and object submission)
• Procedures for content quality and information package quality control
• Object persistence and validation procedures (e.g., checksum identity)
• Metadata policies (including preservation actions and events)
• Procedures and policies for clearing intellectual rights

8

• Storage, duplication and backup
• System inter-operability, security, and data sharing policies
The same survey indicated that only 38% of these organizations reported they had
sustainable funding or some other form of ongoing commitment to the long-term
maintenance of the repository, while the rest did not or did not know. Additionally, the
survey found that institutions felt they lacked organizational expertise much more than
technological expertise. This is also reflected in the fact that nearly 65% of participating
organizations felt the biggest threat to their digital materials was the lack of a preservation
policy.
Li and Banach (2011) later repeated this survey in modified form. While direct
comparisons are impossible, they found a higher percentage (63%) of organizations had
sustainable long-term funding secured, and 66% of respondents reported that they were
currently formulating long-term preservation policies, with a mere 16% indicating they had
full preservation compliance policies already implemented.
Thus, preservation cannot merely be viewed as an issue of enabling and configuring the
correct software implementation. Wilczek and Glick (2006) write:
It seems obvious that no existing software application could serve on its own as a
trustworthy preservation system. Preservation is the act of physically and
intellectually protecting and technically stabilizing the transmission of the content
and context of electronic records across space and time, in order to produce copies
of those records that people can reasonably judge to be authentic. To accomplish
this, the preservation system requires natural and juridical people, institutions,
applications, infrastructure, and procedures.
While the TRAC audit takes very concrete steps to ensure that the software system is
serving the strategic purpose of long-term digital preservation, it seems digital library
managers have a lot of work ahead in defining a common set of high-level organizational
policies and concrete organizational implementation plans for ensuring the repository’s
long-term institutional viability.
Repository infrastructure
The selection of a repository must be in response to the underlying archival needs of the
organization. These should be aligned quite closely with preservation policy, and should
be developed in tandem. Traditionally, digital library infrastructure has been developed
between network information, computer and information and library science professionals,
each field operating upon a different set of philosophical assumptions. The trade-off of
managing the longevity of materials while keeping up with the astonishing pace of evolving
hardware, web, and data-sharing standards implies an “architectural moving target”
(Suleman & Fox, 2001).
Recent studies such as the JISC funded Repository Support Project15
(2010) and that by
the National Library of Medicine have undertaken studies comparing repository software.
The former compares specific repository capabilities against one another, and the latter

15
http://www.rsp.ac.uk/start/software-survey/results-2010/

9

compares features in context against local functional requirements. The ability to preserve
the fixity of a digital object is key here. Jantz and Giarlo (2005) are explicit in their
description of the architectural attributes required for the creation of a trusted technological
framework upon which a long-term repository of digital objects can be built. They define a
digital object as:
A basic unit of both access and digital preservation and one that contains all of the
relevant pieces of information required to reproduce the document including
metadata, byte streams, and special scripts that govern dynamic behavior.
Firstly, this conforms to the Research Libraries Group (2002) definition of digital
preservation cited above. It 1) ensures the “long term maintenance of a byte stream
(including metadata) sufficient to reproduce a suitable facsimile of the original document”,
and 2) allows for “the continued accessibility of the document contents through time and
changing technology.”
Secondly, the digital object’s persistence over time requires that its storage and retrieval
exist independent of any particular software architectural framework. This is what Suleman
and Fox (2001) mean by an “architectural moving target.”
The Open University Library had little trouble coming to a conclusion about using the
Fedora (Flexible Extensible Digital Object Repository Architecture)16
digital repository
framework. The requirements listed here severely limit the choice of a trusted digital
repository.
We are especially guided by the work of the London School of Economics and Political
Science (LSE)17
and their choice of repository based on their own contextual analysis of
factors (Fay, 2010). This study compared DSpace 1.6, EPrints 3.2.0 and Fedora 3.3.
DSpace and EPrints have strengths for open access publication databases, something
The Open University Library already has in place (ORO: Open Research Online18
, which
uses EPrints). These are monolithic repositories that package multiple functionalities into
one piece of software (i.e., they are non-modular), but which do not provide functionality
for born-digital archives and digitized materials, both fundamental to the goals of OUDA.
Fedora has a flexible and extensible repository core that can be customized to local
context, often with additional modular software add-ons (e.g., Solr indexing, Fedora
GSearch, Mulgara triple store, and a Zend PHP web application). This modularity ensures
continual software independence, even if set-up costs are significantly greater. This
reduces chances that the repository will become another “silo” that cannot be interfaced
with existing library and university systems through relational metadata and RESTful
access points.
Additionally, Fedora’s stores digital objects independent of the repository itself, and from
these objects alone Fedora can be rebuilt. This is critical for preservation purposes.
Fedora, in contrast to DSpace and EPrints, also allows for complex object types, persistent
identifier schemas, bitstream preservation tools, customizable ingest workflows, and RDF
relationship data and search capabilities (see Fay, 2010, for further explanation).

16
http://www.fedora-commons.org/

17
http://digital.library.lse.ac.uk/

18
http://oro.open.ac.uk/

10

Marketing OUDA
Gupta and Savard (2010) present an outline of how libraries have had to adapt over the
years to different conceptions of what a library collection and a service model constitutes,
and thus have had to adapt their marketing strategies accordingly. One recent upshot is
that libraries no longer serve as the default go-to resource for information discovery and
research, especially in the last 10 years with the rise of Google and the Net Generation (Mi
& Nesta, 2006).
For instance, an OCLC survey shows that while 45% of university students agree that
libraries provide useful information, a mere 2% start their research using a library web site.
In a comparison of search engines and libraries, the former were preferred for reliability,
cost effectiveness, ease of use, convenience and speed; libraries were preferred for
credibility and accuracy of information (De Rosa et al. 2005).
There are many other reasons for not planning or using a digital repository, chief among
these are the costs, its learning curve, technical impediments to infrastructure deployment,
copyright concerns, organizational adherence, and content duplication and integration with
other existing systems (see Davis & Connolly, 2007).
Librarians may want to shout out “build it and they will come,” but this is likely a poor
reflection of what really happens when trying to establish a digital repository within an
organization or academic institution. Digital repositories will not sell themselves, regardless
of how robust the infrastructure is for preservation and workflow, nor how much they
conform to OAIS or TRAC auditing standards. For this reason, again, we must concern
ourselves with the high-level policy concerns.
Collier (2010) has published an edited book on business planning in digital libraries,
something he sees as rather neglected relative to technical and metadata development.
He defines this in the following way:
Business planning for digital libraries is here defined as the process by which the
business aims, products and services of the eventual system are identified, together
with how the digital library service will contribute to the overall business and mission
of the host organizations. These provide the context and rationale, which is then
combined with normal business plan elements such as technical solution,
investment, income expenditure, projected benefits or returns, marketing, risk
analysis, management and governance.
Thus, having full library and university cooperation and business support for a digital
library’s long-term technical, financial, managerial, and administrative sustenance is
essential.
Marketing and Communications Strategy
Heleen Gierveld (2006) argues that the development and management of a digital
repository depends on strategic social marketing tools that communicate by informing and
explaining, educating and stimulating, involving and inviting, and attracting the attention of
stakeholders and potential users. She stresses two of the “8 Ps” of the service marketing
mix (Wirtz, Chew & Lovelock, 2012), product and promotion, and provides a framework for

11

how to promote a digital repository as a product, even if as an intangible product (see also
Ferreira, Rodrigues, Baptista & Saraiva, 2008). In doing so, she outlines four-strand
communication strategy that we adapt here. It includes:
1) Consultation strategy: collecting feedback information regarding stakeholder
requirements, and engaging stakeholders regarding their domain-specific needs.
2) Pull strategy: attracting engagement and use by offering incentives, making it
attractive, informing users of practicalities of its use.
3) Push strategy: communicating the positive effects use and engagement will bring
about, and encouraging conditions that involve all relevant stakeholders to
participate (e.g., integrating the repository with existing systems).
4) Profiling strategy: using traditional media (websites, brochures, newsletters etc.) to
convince, educate, and raise awareness.
The most immediate and important strategy is university consultation. This is a two-sided
process. One the one hand, consultation represents the need to develop and cultivate
strong relationships with the relevant stakeholders in order to learn about what uses they
envision having for the product, and how they might be able to utilize your product and
potentially integrate it across the organization (Henderson, 2005). This allows a way to
push potential positive effects of participation by eliciting (pulling) their involvement and
making its realization an attractive prospect through the use of profiling strategies, such as
internal newsletters, blogs, posters and talks delivered to relevant stakeholder groups.
In doing so, we also stress that marketing is the tool used to justify expenditures and
costs, IT support for hardware servers, cooperation on preservation policies, and
institutional integration of the repository into the core business processes.
Stakeholder involvement
The first order of business in planning the repository is identifying and analysing the needs
of existing stakeholders. For the initial phase, our stakeholders will be exclusively internal
users. We will consult high-level managers and administrators, research staff and
lecturers, and content producers in the Open Media Unit (OMU) and Learning and
Teaching Solutions (LTS) in order to survey their needs, concerns, and possible ways of
integrating the service with other existing digital lifecycle and workflow processes.
Much of marketing is the creation of particular perceptions. We must create the perception
of university library efficiency and effectiveness, recognizing that different stakeholder
groups have varying perspectives on what this might mean (Cullen & Calvert, 1995). In so
doing, we will segment their various domains of expertise and organizational purpose, and
survey the potential value having a repository of legacy material available might add to
their workflow and digital lifecycle processes.
We must also create the perception of trust. Van House (2002) emphasizes that data
sharing enabled by digital networking technologies implicitly hinges on trust. Trust in the
authority and credibility of data when accessing it, and trust that potential users will not
misuse data after accessing it. As mentioned above, users turn to libraries foremost
because they believe them to be credible and accurate. Trust in repositories is a main goal

12

of the TRAC audit outlined above, and our compliance serves as one of the main goals
when we evaluate our outcomes of OUDA (for issues of repository trust, see Steinhart,
Dietrich & Green, 2009; Ross, 2012; Prieto, 2009).
Profiling for feedback and evaluation
The second order of business in creating a successful repository will be to properly profile
the potential service. With any technical project, this process must be very wary of the
various levels of technical understanding. Maintaining IT support for servers and storage
systems requires a different language than speaking with high-level managers in
justification for continued financial support.
The consultation process itself will provide the fodder for planting the seeds of product
promotion. That is key here. The consultation process for information infrastructure
projects must be seen as an iterative process that occurs over the lifespan of the project
(Schwalbe, 2007). As such, each consultation process is an opportunity to market the
evolving nature of the product, and sell its continued support. For instance, before
surveying stakeholder opinion regarding how a digital repository may or may not
complement and/or add value to their workflow, we must profile—in the form of a talk,
poster, presentation or other such media—our plans for developing the infrastructure, and
in the first instance how it is constrained first and foremost by the needs of trustworthy
preservation. Later, after having integrated the results, a second consultation will profile an
improved prototype that will again serve to attract (pull) the relevant stakeholders and
encourage (push) them to maintain allegiance to the product.
For this reason, when discussing requirements with content producers, having various
materials prepared for presentation will help them envisage the kinds of service we intend
to prototype. At first, this may include only mock-ups of workflow; later, it might include
actual prototypes for how this workflow operates in the chosen repository architecture (i.e.,
Fedora). When a producer is faced with creating a new course, being able to view a
prototype of how the repository will hold a few years of well-organized legacy course
materials (and the digitized books, video, audio, and images making it up), s/he will see
potential value to their own workflow in re-using digital material as well as providing a
model for inspiration when viewing how successful legacy courses were organized and
conducted.
Profiling developmental plans and prototypes thus serves a three-way purpose. It creates
discussion and interest in their continued support. It also provides a way to illustrate earlier
feedback has been integrated, thus reinforcing the notion that their continued participation
and feedback is important. Additionally, it provides a forum for continuous evaluation. That
is, often their feedback will be operationalized as a series of interim outcome measures.
We’ll discuss this more in the next section. Thus, each and every stage of evaluation (see
below) must be treated as an exercise in marketing and promotion.
The university provides plenty of opportunity for showcasing products. Thus promotion will
not be limited to consultation sessions intended for evaluation. Promotion will also occur
through formal and informal meetings, talks, poster sessions, as well as the publication of
reports, newsletters and blogs (see Ferreira, Rodrigues, Baptista & Saraiva, 2008).
Infrastructure and processes for this are well established at the Open University.

13

Assessing OUDA
Our marketing plan attempts to integrate promotion into the process of developmental
prototyping and evaluation. By promoting the service, we intend to educate and inform
stakeholders of how the repository’s continued development will add value to their unit.
Through this promotion, we also stay tuned into whether our developmental process is
usable and useful.
However, it is its evaluation and assessment that will provide objective measures of its
usability and usefulness to internal stakeholders. For instance, whether the repository
serves to facilitate course production, decrease overall costs (by increasing re-use and/or
decreasing duplication), and increase student and researcher satisfaction. These are
candidate outcomes that we can target as developmental milestones in promoting its long-
term continued service at higher levels of management (see Rubin, 2006), such as
enhancing university reputation and image, and providing a new set of trustworthy access
points that demonstrably decrease the timeframe and overall costs of producing and
managing an online course.
There are two major criteria of success for this digital library developmental phase. As we
mentioned in the introduction, they are:
A. Preservation policy and repository infrastructure
B. Access (usefulness and usability) and content re-use
The first criterion, preservation and its support through robust and trustworthy
infrastructure, was delineated in the planning section above. We will treat it only briefly
again below, in terms of its value to assessing outcomes. The second regards a set of
criteria that must be operationalized with respect to how usable and useful stakeholders,
as future users, perceive the repository in terms of its future value to their workflow. That
is, while this first phase (2 years) should allow time to implement the repository
architecture, it will not be until the next phase that it will be properly populated with content
for actual use. This will first involve amongst other things setting up a cataloguer’s
interface for ingest workflow and quality control processes, deprecating the disparate
legacy containers the content was stored in, and implementing new library processes
around the digital repository. That will involve deeper levels of change in culture from
traditional library collection development to one adapted around content management to
maximize use of existing and emerging internal collections to support users in formal and
informal learning (Mi & Nesta, 2006). That is, adapting it to the realities of the net
generation.
The OUDA evaluation framework: The triptych model
There is a lot of research devoted to the study of digital library evaluation criteria.
Chowdhury and Chowdhury’s (2003) book on digital libraries is a good place to start. One
constant consideration for a university library is to provide for users with special needs
(Kwak & Bae, 2005). Related to this is research by Inskip, Butterworth, and MacFarlane
(2008) that has found that usefulness and usability is highly dependent on the user’s level
of research experience. Thus, while OUDA target audience in the initial phase will be

14

content producers—or faculty and staff within the university, presumably with higher levels
of research experience, our long-term goals include students and to some extent the
general public. Even so, nobody wants to use a difficult system, and the OU does aspire to
Google-level ease and simplicity in its search and retrieval functions (see again Mi &
Nesta, 2006). For instance, one study by Kengeri, Seals, Harley, Reddy, & Fox (1999)
found very few differences between novice and experienced users of digital libraries, and
Theng et al. (2008) even noted that children designed systems incorporating the same
usability features that adults typically desire.
Another set of resources comes from DELOS19
, a “Network of Excellence on Digital
Libraries,” led by Norbert Fuhr. It contains an excellent set of evaluation studies and an
annotated bibliography of articles relevant to the strategic guidance on issues of
usefulness and usability.
More specifically, Fuhr et al. (2007) developed a digital library evaluation framework by
surveying a range of previous evaluation research. Through an analysis of factors, he
summarizes three dimensions important for evaluation, the first being the system and
technology, treated above. The second is data and the collection, or what we are calling
the usefulness of the system. That is, it addresses questions of the quality of content and
its metadata and how well it can be managed and accessed. The third relates broadly to
the kinds of users it targets and their information seeking tendencies and motivations
(“usage”), what we are again calling usability. They have named these dimensions the
“interaction triptych model,” reproduced in Figure 2 below:

Figure
2
Many authors have generated a similar set of abstractions, prior to the studies of Fuhr, et
al. For instance, Borgman (2013) mentions three components, easily mapped onto this
model, that are key to information access: connectivity (how effectively systems deliver
content), content and services (how users interface with content, i.e., usefulness), and
usability (how users interact with a system). Xie (2006) also collected evaluation criteria
identified by users themselves, and categorized these in five ways, not at all dissimilar to
the approach we are using. They are: usability, collection and service quality, system
performance, and user opinions.
This interaction triptych framework will thus serve as the basis for assessing and
evaluating OUDA as a digital repository system. Each of the three axes of evaluation,
performance, usefulness and usability, will serve as an outcome that we will assess by

19
http://www.delos.info/index.php?option=com_content&task=view&id=26&Itemid=52

15

operationalizing its parameters for study. An assumption we will develop in our
assessment section is that user satisfaction along these dimensions will increase re-use
and decrease content duplication.
Performance
The most important of these for this first phase is how well the system interacts with the
formats, structures and representations of the digital content it is intended to preserve.
While most of the hardware and server systems fall within in the domain of university IT
services, there is another component left to library services. The TRAC (Trusted
Repositories Audit & Certification) criteria and checklist will serve as our primary outcome
measure for determining the performance of the system when interacting with content.
TRAC is primarily an evaluation tool for determining repository trust (see Steinhart,
Dietrich & Green, 2009; Ross, 2012) for long-term preservation. However, it is broken into
three sections:
A. Organizational Infrastructure (governance, staffing, policy, licensing, and financial
sustainability)
B. Digital Object Management (ingest procedures, preservation storage and access
management)
C. Technologies, Technical Infrastructure, & Security
Sections B and C deal extensively with criteria of hardware and software (Fedora)
implementation (C) and how different types and forms of digital content should be created,
formatted, structured and described for ingestion as a sustainable information package
that can both interact with said software system (B), but is also preserved independent of
any particular system for long-term preservation purposes (see RLG-NARA Digital
Repository Certification Task Force, 2007)20
. While section B also focuses on parts of the
ingest process that will only be relevant to a later stage of OUDA development (e.g., the
development of a cataloguer’s interface), many aspects of its dozens of criteria-points are
relevant to this outcome.
There are other definitions of performance evaluation that, while certainly relevant to
system performance, will fall outside our scope of evaluating outcomes. They mainly deal
with precision and recall factors taken from studies in information retrieval. They, along
with measures of usefulness and usability, were operationalized and studied for their inter-
factorial influence on one another (Tsakonas & Papatheodorou, 2008).
While the specific results of this study fall outside our scope, the way they operationalized
the three axes are relevant to defining how we will measure the outcomes of OUDA.
Below, in Figure 3, we reproduce the edited triptych framework. Beside the axes of
performance, usability, and usefulness, Tsakonas and Papatheoudou list the attributes
they summarize as the most important indicators for measuring each.

20

16

Figure
3
Usefulness
The usefulness of user-content is a reflection of “how users perceive the relevance of a DL
[digital library] with their needs, the width, the breadth, the quality, as well as the validity of
its collection, and the ability to serve their goals” (Fuhr, 2007). There are two methods of
assessing this: user-studies, and user behaviour. User behaviour comprises the
interdisciplinary investigation that involves many applications from cognitive science and
psychology that attempt to assess how different kinds of humans seek information, how
their cognitive states such as processes of motivation, reasoning, intuition, luck and user-
eye and mouse movements play out in their information choices (see Kuhlthau, 1991;
Jeng, 2013). These factors will be relevant only to later phases of OUDA development.
For now, we shall employ user-studies to assess what stakeholders would like to see
OUDA support in terms of their information needs and preferences. For instance, how can
OUDA deliver relevant and reliable content in a format and at a level appropriate to their
needs in producing content for the OU course modules? These needs will again be
assessed in an iterative manner, as mentioned in the marketing section above. That is,
initial studies will determine need and preferences based on the underlying goals of the
preservation repository, and later studies will evaluate whether iterative prototypes spaced
over developmental time (presented in the form of posters, talks and presentations) are
actually progressing in a way that are perceived to actually serve the needs and
preferences they identified. User studies use a variety of techniques such as surveys,
focus groups, questionnaires and online forms (Fuhr, 2007; see also the DELOS
framework).
Usability
Usability is a major field of study in its own right, the output of which far exceeds in
quantity and scope that produced by information science researchers. Entire fields of
study such as information architecture, human-computer interaction, and user-experience
and interaction (UX/UI) maintain a lively community of usability discussion and research.
Web developers and designers are often the most passionate and knowledgeable

17

individuals on issues relating to usability. For instance, the website A List Apart maintains
an excellent set of resources for developers21
and is usually the first to document
innovations such as responsive design in practical ways.
The attributes listed above in the triptych model—ease-of-use, aesthetics, navigation,
terminology, and learnability—do a good job at triangulating what is at issue in usability
studies. The International Standards Organization (ISO) defines usability as “The extent to
which a product can be used by specified users to achieve specified goals with
effectiveness, efficiency, and satisfaction in a specified context of use”22
. Usability testing
is often based on principles of iterative design, and its methods of evaluation are various.
For instance, OUDA will employ various methods for designing the interface and its
organizational structure from scratch. This will employ methods such as card-sorting, focus
groups, and surveys. Later, when iterating through actual design prototypes, we will use
methods such as cognitive walkthroughs, and thinking aloud (see Rousseau, Rogers,
Mead et. al., 1998).
An earlier study by Tsakonas and Papatheoudou (2006) found that users prefer a system
that is useful over usable, insofar as its primary goal is to locate content within a system.
However, research by Xie (2006) found usability ranked higher, so one must conclude the
outcome is heavily influenced by methodological artifact. However, the former outcome
corresponds closely with De Rosa et al.’s (2005) study mentioned above. That is, that
users usually only turn to library sites if issue of accuracy and credibility are at stake; that
is, the usefulness of the content. However useful it may be, it is well established (Joint,
2010) that users prefer one-stop aggregated search features over gateway databases and
federated digital libraries, something the librarians still have not come to terms with when
developing digital systems that overwhelm most users with unnecessary metadata filters
and advanced search features they assume specialists prefer. Tsakonas and
Papatheordorou (2008) note that if these kinds of features are developed, their ease of
use is absolutely essential to user satisfaction.
Conclusions
This literature review has attempted to outline many of the most important considerations
when planning, marketing and assessing a digital library repository for the long-term
storage, preservation and dissemination of university online-course material.
Our chief goal for Phase One planning (a two-year plan) is encompassed in the statement
below:

21
http://alistapart.com/topic/usability

22
"ISO/AWI TR 9241-1". International Organization for Standardization.

18

At its core, OUDA will provide a one-stop shop to store and preserve all legacy course
material, and incorporate new course materials, instead of this content being stored in a
slipshod manner across various OU faculties responsible for its production and
presentation (e.g., researchers, lecturers, and course production units).
Most stakeholders are going to want well-defined reasons and evidence outlining what the
long-term benefits are to adopting a digital preservation repository project. This will involve
carefully measured and marketed indicators such as cost reductions, speed and efficiency
increases to organizational workflow, repository effectiveness at improving the overall
quality of its services, as well as how it will enhance the organization’s reputation. These
indicators will need to be operationalized, measured, and evaluated against expectations
based an organization’s strategic mission.
In the planning, marketing and assessment sections that follow, it is our responsibility to
show in concrete ways how OUDA will add to the university’s strategic mission of
increasing educational value to its virtual customer base, and providing researchers and
digital learners with high quality, low-cost and trustworthy content through the use of open
and inter-operable standards, making education easily accessible to all.

Planning OUDA as a preservation repository
Introduction
The Open University Digital Archive (OUDA) is our solution to providing core information
and enterprise content management systems to service the needs of the Open
University’s online educational model and corporate e-business. The planning
documentation below will discuss the business model project plan for OUDA and how it is
to satisfy both library and university strategic planning.
OUDA is in Phase One of planning. Phase One is a two-year plan. This first phase
describes the implementation details for a developmental prototype of a digital repository
to archive and preserve university generated content. Its plan for development is in
response to increasing demand from Open University (OU) stakeholders to provision
content management services for university-generated material that requires long-term
sustainable access and preservation management.
Relationship to library strategic planning
Purpose
The vision of OUDA aims to enhance the visibility and academic reputation of the Open
University (OU) by preserving selected Open University historical, learning, teaching, and
research content. Exposing this learning and research material will demonstrate the quality
and increase the usefulness of OU’s pedagogical methods and illustrate how they have
developed over time.
Because these resources have been expensive to produce, there is a growing demand
from faculty, academic researchers, university content producers, and library archival
services to address the long-term sustainable management of this growing corpus of
legacy material. For this reason, OUDA will operate as an important repository for the
preservation, discovery and re-use of OU resources. The development plan of OUDA must
encompass digital preservation policy, provisioning for digital services, content licensing
and rights, as well as the technical and infrastructure requirements in relation to preserving
and managing access across stakeholder groups to legacy content and materials.

20

The strategic context
As a world-leader in distance learning the Open University has moved rapidly into the
world of digital content. Our reputation as a university is built on our digital content
services. Many millions of pounds are invested yearly in creating digital content and digital
services to deliver innovative and effective learning.
OUDA will be a space where collections of material from the OU’s rich history can be
discovered and preserved. As a ‘digital university’ we need to be taking steps now to
preserve the best of the OU’s digital material. If we do not act now material will continue to
decay or be lost.
OUDA will be developed in alignment with the OU’s mission and core value statements. Its
values are inclusivity, innovation, and responsiveness23
. Library Services strategic
priorities (2010-2014) have been developed in accordance with these values. In regards to
OUDA, the most relevant of these strategic priorities is to:
• Focus direction on the virtual customer base ensuring that resources, systems and
processes are developed in line with the distributed and in future global nature of
this complex and fragmented customer base.
• Support researchers in exploiting their use of their own content and facilitate their
access to other quality research resources and networks.
• Provide stewardship and strategic advocacy for sustainable digital preservation of
teaching materials and research data management services for long-term access
and reuse.
Broadly envisioned, OU Library Services have developed a set of strategies nested within
the broader OU strategic framework. The following OU Focus Area Objectives have been
identified as an indispensible set of objectives that OUDA must meet if it is also to meet
the goals of university and library strategic planning. These are:
Focus Area 2: Learning and teaching efficiency
To improve upon the efficiency of course production, presentation and assessment
in order to reduce the cost and improve the effectiveness of the core business
Focus Area 3: Developing pedagogy
Continue to innovate and develop pedagogy to maintain and enhance the OU’s
reputation for quality innovative teaching and learning
Focus Area 5: Research and scholarship
The improved promotion, dissemination and impact of OU research and
scholarship, and its integration with course production, presentation and
assessment

23
http://www.open.ac.uk/about/main/files/aboutmain/file/ecms/web-content/strategic-plan-2012-15.pdf

21

Focus Area 6: Business efficiency
Financial sustainability through the delivery of good value by facilitating university-
produced content discovery, decreasing course material duplication, and increasing
its re-use, thus leading to a greater return on investment and helping to save
production costs
Relevant Literature
One of the most important results of the research we reviewed was the establishment of
the TRAC (Trusted Repositories Audit and Certification24
) standard, developed by the
NLG-NARA Digital Repository certification Task Force (2007). It serves as an authoritative
checklist of the necessary evidence required to demonstrate long-term repository viability.
Its three sections, which will constitute a principle primary outcome measure of repository
trust (see Steinhart, Dietrich & Green, 2009; Ross, 2012) for long-term preservation, are:
sustainability)
management)
Fedora was chosen as our repository software system. Fedora conforms to the Research
Libraries Group (2002) definition of digital preservation. It 1) ensures the “long term
maintenance of a byte stream (including metadata) sufficient to reproduce a suitable
facsimile of the original document”, and 2) allows for “the continued accessibility of the
document contents through time and changing technology.”
The analysis of repository software published by the London School of Economics and
Political Science (LSE) guided our decision to select Fedora (Fay, 2010). Fedora is
flexible, extensible, and modular. This allows for add-on software independence, and the
digital objects are also stored independent of the repository itself, critical for preservation
purposes. Fedora, in contrast to DSpace and EPrints, also allows for complex object
types, persistent identifier schemas, bitstream preservation tools, customizable ingest
workflows, and RDF relationship data and search capabilities.
However, correct software configuration is not enough to ensure long-term preservation
(Wilczek & Glick, 2006). We must ensure preservation is part and parcel of long-term
university and library strategic planning. That is, while preservation is the goal, OUDA is
also being developed in alignment with the OU’s mission and core value statements
outlined above. That is, the repository should also create value insofar as it addresses the
four OU Focus Statements above. That is, it should 1) make teaching and learning more
efficient, 2) innovate pedagogy to increase OU’s reputation, 3) promote OU’s research
impact, and 3) increase business efficiency in the production of course materials (and
hence decrease costs).
Long-term preservation requires good policy, which integrates people, applications,
procedures, workflow plans, institutional and technical support, and correctly configured

24

22

applications and infrastructure. From the literature review above, we outlined useful
considerations for a strong policy framework (see Beagrie, Semple, Williams et al., 2008;
Seamus, 2012). They are:
• Organizational viability: How will preservation serve organizational need, and who
are its stakeholders?
• How will it be integrated into and how does it relate to other organizational strategic
priorities?
• What are the objectives of preservation and how will these be supported?
• Has the content it will potentially store been defined and delimited? What kinds of
collections will it store?
• Who is procedurally accountable to these policies and what are their obligations?
• Who is financially and organizationally responsible for sustaining the repository?
We also reviewed particular lower-level implementation clauses that should be articulated
within policy documentation. Most of these are covered in the TRAC checklist. They
include (adapted from Kenney & Buckley, 2005):
Collier (2010) has published an edited book on business planning in digital libraries,
something he sees as rather neglected relative to technical and metadata development.
He defines this in the following way:
Business planning for digital libraries is here defined as the process by which the
business aims, products and services of the eventual system are identified, together
with how the digital library service will contribute to the overall business and mission
of the host organizations. These provide the context and rationale, which is then
combined with normal business plan elements such as technical solution,
investment, income expenditure, projected benefits or returns, marketing, risk
analysis, management and governance.
Thus, having full library and university cooperation and business support for a digital
library’s long-term technical, financial, managerial, and administrative sustenance is
essential. To do so, we must first know who these stakeholders are.

23

Stakeholder and user needs analysis
The development of OUDA must be first and foremost customer focused. The customer in
the case of OUDA includes a wide range of stakeholders. We define the following three
groups of stakeholders, and will identify and describe these each in turn:
1) OU Library Stakeholders (internal)
2) University Unit Stakeholders (external)
3) Students and Public Stakeholders (external)
It is these stakeholders that in the first instance also represent our user base. That is, all of
these stakeholders represent potential users, with their own set of needs from which they
can benefit depending on the impact the service is expected to have on these needs.
Thus, within each of the stakeholder/user sections that follow, we will outline the various
kinds of need each of these stakeholder/user groups describe as important to fulfilling their
own strategic goals of satisfying university mission and the informal or formal means by
which these have been assessed and/or observed.
In the same respect, we will also provide a brief analysis of the associated set of risks and
costs of service implementation (or non-completion of its stated goals), as well as the level
of demand we should expect from each of these groups given the benefits and impact the
service is expected to have on their interests.
STELLAR Project: Preliminary stakeholder views assessed
Before identifying and analysing our stakeholder groups, however, it is worth reporting the
results of a relevant study that addressed how semantic technologies might enhance the
lifecycle of learning resources. The study also addressed the value of legacy learning
materials no longer in presentation to students.
The eighteen-month JISC funded STELLAR project (non-published, 2013) run by OU
Library Services surveyed (online) the perception of stakeholders (n=561), which included
asset creators (academic and non-academic), senior administrators, asset managers, and
regular internal users (not including students). Follow up interviews were conducted with
approximately 10% of each stakeholder strata (including six senior stakeholders). The
survey used the “balanced scorecard approach”25
.
While the results themselves are quite nuanced, a few outstanding tendencies were
observed. For instance, 89% agreed or strongly agreed that the maintenance of an archive
of non-current OU material is important to OU reputation, with merely 2.3% disagreeing,
and 75.9% believing it should be maintain in perpetuity. 90% of respondents agreed or
strongly agreed that non-current learning materials are important to the context of higher
education history, and 91% of those involved in course module production agreed or
strongly agreed that they were likely to look to previous material for inspiration or re-use
when producing new OU learning material.
The study concluded that OU stakeholders place high value on legacy learning materials,
whether that mean personally and/or professionally, financially, or as having value to

25
http://en.wikipedia.org/wiki/Balanced_scorecard

24

higher education and/or the internal processes and cultures involved in their production.
This supports the belief that these materials should be preserved, and that doing so will
enhance OU reputation, underscore its history and that of higher education in general,
provide academics and students more opportunity to utilise this rich set of resources, and
facilitate their re-use in course production.
OU Library stakeholders
Based on internal feedback and observations from library staff, one main gap in the
portfolio of services is the provision for long-term preservation. In the planning section in
literature review above, we outlined how preservation policy must be linked to core library
strategy.
Recent decisions have determined that our current cataloguing system, ExLibris’ Voyager
Integrated Library System, no longer serves the needs of the library or university for the
maintenance of the various kinds of legacy digitized and digital-borne multi-media that
require long-term storage preservation. Nor does it allow for the effective structuring and
description of these digital objects through the use of modern metadata and relational data
standards that enable extensible, modular, and inter-operable access to these objects
across various web platforms and web service models.
Additionally, a number of projects (e.g., Videofinder26
) have been funded to build websites
and resource collections without the requirement to develop a sustainability plan and
budget to maintain access to these collections beyond the life of the projects themselves.
Alongside the remit of OU Library Services to archive these materials, there is potential for
reuse of these rich resources in learning, teaching and research. Thus, a service that
brings together the OU Library’s digital collections from these disparate sources will be
welcomed.
The changing role of Library Services involves shifting from being providers of print
services to focusing our services entirely on the digital. Additionally, we must start looking
beyond the management of licenced resources from external venders into a future in
which open resources play a more critical role. It is thus essential that university-created
resources be exploited for longer periods of time and be made more widely available.
The following chart illustrates the set of internal library stakeholders and their respective
stake and set of expectations they hold for the project as well as the potential impact they
will have on the project’s success.
Role Stake / interest / expectations Potential
Impact
Library Director
Nicky Whitsed (NW)
Programme and Project Sponsor. That OUDA delivers a
convincing prototype for a digital library service, and can
increase the value of the library service model.
High

26
http://videofinder.open.ac.uk/ commissioned by Library Services to hold video and audio assets developed across the
university for use in course modules or as co-productions with the BBC programme services. Internal (non-public) use
only, the system is now operated by the OU Open Media Unit (OMU)

25

Associate Director
(Information Management
and Innovation)
Gill Needham (GN)
Steering Group member and executive accountable for
business case, project benefits and outcomes. To ensure
resources are sufficiently available for project success. That
OUDA delivers a convincing prototype and increases the value
of a digital library service. That the project is promoted across
university units (see external stakeholders).
High
OUDA Programme
Manager (RN)
That the project delivers on its aims and objectives as a service
model. That the project is sustainable, scalable, and is
developed in line with its stated strategic objectives. That the
project is promoted across university units (see external
stakeholders).
High
OUDA Project Manager
(LM)
That the project team delivers and meet project aims and
objectives according to stakeholder requirements. Scheduling,
reporting, keeping the project on track. That the project is
promoted across university units (see external stakeholders).
High
OUDA Systems Developer
(JA)
That the technical infrastructure will incorporate appropriate
standards and tools, meet user requirements, and be
documented.
High
OU Archivist (RC) That OUDA is build to appropriate international preservation
standards (e.g., TRAC and DRAMBORA) and offers a
sustainable solution for OU archives and teaching materials.
High
OU Metadata Development
Manager (LW)
That OUDA is built to appropriate international metadata
standards, is developed using linked data principles, and allows
for easy access to OU archives and teaching materials.
High
Library Academic and
Student Support Leader
That users needs are met, and champions support within OU
division of academic and student services.
Medium
Library IT team Technical advice when needed. Support with OU IT Services. Medium
Library IM team Some aspects of content preparation and migration. Quality
control checking. Some aspects of standards and policy work.
Medium
Other library staff That OUDA will allow access to more content and better
services.
Low
University stakeholders
The following chart illustrates the set of external (outside the library) stakeholders within
the Open University. This group of stakeholders will primarily provide advice and
consultation services in order to inform aspects of OUDA project development.
In the third column of the table that follows, we briefly outline what we view to be the
potential need and/or interest the stakeholder might have in establishing OUDA as an
institutional repository. These have only been assessed informally, through observation,
internal question and answer sessions, and feedback and focus group sessions held over
the past year of system prospecting.
In the fourth column, we assess the potential impact that the completion of this project will
have on the respective unit’s workflow and its contribution towards university strategic
goals.

26

Role Stake / interest Potential need / benefit Potential
Impact
Pro-Vice Chancellor
of Learning and
Teaching
(PVC-LTQ)
Professor Belinda
Tynan
Funding and university project
sponsor. University learning and
teaching strategy and
governance, responsible for:
• Learning and Teaching
Solutions (LTS)
• Library Services
• Open Media Unit (OMU)
Value added to university
workflow. Enhanced university
image and reputation across the
UK and the world. A historical
timeline of university content
preserved over the long-term,
reduces risk of content loss. Saves
money.
High
LTS (Learning and
Teaching Services)
Publishing services. Supports
the development, production,
and delivery of distance learning
materials.
Course module re-use and non-
duplication. Integration with VLE
(Moodle) and course development
workflow. A single e-production
system.
High
OMU (The Open
Media Unit)
Managing the production of
OpenLearn, YouTube, iTunes
and BBC output to support the
OU’s social and business
mission.
Searchable audio-video system
record to replace Videofinder and
expose content. BBC and iTunes
content can appear in OUDA, and
vice-versa. Linking content
between data silos.
High
KMi (Knowledge
Media Institute)
Supports linked data and
semantic web technologies.
Including our RDF namespaces:
www.data.open.ac.uk.
Little value to KMi. But continued
KMi development of linked data
minting and mining highly
important to semantic sharing.
Low
OU Central
Academic Units
Academic units responsible for
writing, teaching, and course
delivery.
Easy reference/access to legacy
courses and how they were
presented. Exemplars of good
practice. Academic reputation. It
fosters non-duplication and re-use
in teaching materials.
Medium
OU Information
Technology
Server infrastructure for OUDA.
Long-term digital storage.
Closer to becoming single system;
inter-operability.
Low
Students and public stakeholders
The following chart illustrates the set of external (outside the library) stakeholders from
beyond the borders of the Open University. This group of stakeholders will provide mostly
advice and consultation services in order to inform aspects of OUDA project development.
Noteworthy here is the stake existing professional communities will have and how their
interest and expertise in issues of higher education, preservation, and linked data will play
a part in the development of OUDA. For more information regarding these professional
user communities, especially those of preservation and linked data, please re-visit the
literature review at the beginning of this report.

27

Role Stake / interest Potential need / benefit Potential
Impact
OU Students Potential future users of
OUDA. Prospective and
current students can taste OU
course history. Past students
can access old course
material.
Access to a preserved record of past
course module material. A better,
more integrated online experience.
For prospective students, especially,
evidence of how OU online study
works.
Low
Wider HE and
library community
(academics and
staff/students from
other institutions)
Potential future users of
OUDA. Publicity and support.
We will seek constructive feedback
from these potential future users.
Low
Linked data
community (when
data is exposed in
RDF format)
Potential future sharers of
OUDA linked data content.
from these potential future users, as
well as advice and guidance in
implementation.
Medium
Digital preservation
an curation
communities
Interest in what we are doing.
Publicity and support.
from these potential future users, as
well as advice and guidance in
implementation.
Medium
General public May eventually want to use
OU resources for free.
None at present. Low
Stakeholder benefits, costs and risks
General benefits
• There will be a location to show non-current digital material from the OU, to allow
the Open University to preserve and present the OU’s ‘digital heritage’, ‘institutional
memory’ and ‘student experience’
• For the first-time it will be possible to show the full content of an historical module in
digital form to staff
• It will be much easier to search and access digital content from the OU’s archives
collections
• Collections of material such as images will be much more discoverable and visible
• There will be the potential to use the linked data elements of OUDA to link to and
from other content
• The University Archive will become a much more visible service
• There will be a digital preservation service, making it clear to stakeholders how they
can go about ensuring that their content can be preserved.

28

• OUDA will support linked data as part of routine activities, using this technology
within Fedora (with an RDF triple-store) and as a tool to display content from other
SPARQL endpoints (e.g. from data.open.ac.uk)
Costs and risks
For this reason, library stakeholders have established a case to justify the need for this
digital repository. Certain types of content are at risk if no plan is put in place for continued
management. This includes the OU’s born-digital and digitised teaching output and assets.
That is, the risk of not implementing some plan to develop a digital repository to store and
preserve this content is now reaching a critical point. Non-action is not an option.
Additionally, the pace of university e-content development is outstripping the library’s
capacity for its effective organisation for future accession. This is true especially given the
quickly changing business model in higher education. A digital repository that is flexible,
extensible, modular, open and agile is essential to support a university’s e-portfolio
development and maturity in an increasingly competitive industry based on ever-
quickening changes in software and web systems.
There are various costs associated with developing this repository. The main one being a
re-structuring of library processes to accommodate this new model. Service delivery may
suffer during development, as will the replacement and retraining of staff to accommodate
the new service delivery structures and procedures.
The external group of stakeholders, with the exception of PVC-LTQ, may potentially view
OUDA as a threat to their existing services. Care must be taken to convince these groups
as part of our marketing and communication plans that OUDA will seek to complement
their existing digital architectures and services, not replace or disrupt them.
One additional risk is that this complicates the workflow processes of other units, insofar
as it is not adopted wholesale for digital preservation of non-current course materials, but
instead becomes just another option amongst many for where these materials end up.
Demand and impact
We estimate high demand from internal library users. Currently, the fragmented and short-
term approach to digital storage creates large amounts of frustration in staff. With too
many overlapping local systems, each one replacing the next as software changes,
content resides across various “silos” each requiring a different set of work processes for
its discovery, access and retrieval. It is non-persistent and without common policy for its
long-term preservation. Establishing these kinds of changes will have high impact for
library users with a stake in digital content systems, including metadata librarians,
archivists, digital project managers, systems developers, cataloguers, and academic and
research support specialists.
We estimate high demand from most external stakeholders, with the proviso that the risks
associated above (and the limitations delineated below) are managed and controlled. The
usefulness and the usability of the service architecture must also be very high it the
repository is to be utilised across university units. This will ensure that the repository is

29

trustworthy, clearly and capably administered, accessible, and complies with its stated
intentions in user-friendly ways.
Recommendations for action
Focus on the customer. Focus OUDL services and solutions on the needs of core
customer groups: currently registered OU students, teaching staff, ALs and research staff.
Convene a LTQ Digital Libraries Development Working group with cross-university
representation to secure broader OU stakeholder involvement and engagement to drive
OUDA forward.
Engage the OU community with OUDA proposals to ensure they meet expectations and
service requirements, for guaranteed levels of high quality, trusted and sustainable
services and resources.
Capitalize on existing semantic web technology expertise resident in KMI to prototype and
test scalability and adoption of the OUDA framework, standards and technologies.
Work closely with the LTS e-production staff to deliver potential business efficiency
savings associated with OUDA, ensuring scalability and robustness of OUDA in particular
relating to implementing sustainable good practice in information, digital asset and
metadata management.
Benchmark the OUDA framework and systems strategy against world-class digital libraries
currently in service. In particular, the China Academic Digital Library Information System
(CADLIS) model now serving 1800 universities across China.27
Our chief goal for Phase One planning, marketing and assessing this prototype is
encompassed in the statement below:
The Open University Digital Archive (OUDA) has the following goals:

27
Wenqing, Wang and Ling, Chen (2010) Building the new generation China Academic Digital Library Information
System (CADLIS): A Review and Prospectus. DLib Magazine, May/June, Vol 16, No 5/6
http://www.dlib.org/dlib/may10/wenqing/05wenqing.html

30

• Capture OU history, learning, teaching and research materials to enable their
preservation, discovery and reuse, inform pedagogical decisions and facilitate
educational research.
• Complement existing platforms (such as OpenLearn, iTunesU and Study at the OU)
to support students’ informal to formal learning. Where assets already appear online
OUDA will drive traffic to the relevant platform for access through the use of
semantic technology, whilst aiming to preserve a high quality copy and associated
metadata in OUDL for the long-term.
• Contain a selection of fully-searchable digitised and ‘born digital’ OU materials from
videos and images to digitised documents; thus providing access to archive
materials previously only accessible by visiting the OU Archive, and supporting the
enquirer and study experience.
• Manage and preserve digital content in a long-term, sustainable manner in
accordance with OU policy and international preservation and metadata standards.
The following are the six outcomes according to which we will measure the success of
OUDA (see the assessment section) after this initial phase one of planning:
1) A preservations policy plan is in place and has been approved by the OUDA
Steering Group. This preservation policy plan will contain all of the elements
described in the literature reviews above, and will be benchmarked against other
successful institutional policy directives.
2) OUDA will comply with the components of the TRAC audit that fall within the scope
of Phase 1 planning. That is, TRAC Sections A (Organizational Infrastructure) and
C (Technologies, Technical Infrastructure, and Security). Section B contains
elements of cataloguing ingest workflows, and will belong to Phase 2.
3) OUDA will contain prototypes of various content types. This will include complete
records and content of course material (i.e., video, audio, texts, supplementary
materials, books) for at least 3 non-current online legacy courses. This material will
cohere as part of a course, but also as materials that cohere as collections in
themselves (e.g., a collection of BBC videos, each video which is also “part of” one
or more course modules, or not).
4) The OU Archive study material records (metadata) are discoverable via OUDA.
They allow for effective searching, browsing and faceted filtering to discover content
and content types. That is, the content becomes useful to a user insofar as it is
relevant, in a readable format, reliably discoverable, and is at the correct level and
coverage specified.
5) OUDA can be used to guide people towards content related to their search on other
platforms such as iTunes-U and OpenLearn using linked data functionality. That is,
it will have linked data functionality through a SPARQL endpoint and interact with
the URIs minted for course materials at data.open.ac.uk.
6) OUDA has incorporated feedback from its most relevant external stakeholder
groups. That is, the OUDA prototype has received positive feedback from

31

stakeholders across groups regarding its potential ROI for increasing course
material re-use and course production productivity. And its potential value and utility
for enhancing OU’s academic and research reputation.
Long-term outcomes that fall outside of the scope of Phase 1 planning are as follows.
OUDA success also hinges on its ability to transition to Phase 1 goals and outcomes.
Some of these include the following:
• The OU Archive is able to use the OUDA as their main cataloguing environment for
study materials.
• OUDA is the main interface for digital preservation workflows and processes.
• Library management is able to use usage reports generated by OUDA to make
informed business decisions.
• Archive staff is able to use preservation reports generated by OUDA to make
preservation decisions.
• The OU Archive is able to offer a digitise-on-demand service for video and audio
content to staff through OUDA.
• The OU Archive is able to offer a service to OU staff to preserve and make
accessible their digital content, if it meets selection criteria.
• OUDA is used to promote the OU Archive, Library Services and the OU itself to the
wider public.
Responsible parties
Internal OU Library Services has developed a hierarchy of responsible parties that is
structured all the way up to the Director of Library Services, Nicky Whitsed (NW). She
chairs the steering group of internal library stakeholders that formalize all decision-making
processes undertaken by the library as a university unit. The steering group is made up of
representatives of the three major library sub-units: 1) Academic and Student Services, 2)
Business Performance and Management, and 3) Information Management and Innovation.
The first sub-unit (1) ensures that the project is well represented across academic and
student services. It ensures the library is working closely with course module production
teams and the various faculty branches that make up the academic heart of the university.
The second sub-unit (2) addresses various system support issues that occur within the
library. While OU IT Services hosts OUDA infrastructure, the library has a small technical
team that provides back-up expertise and support to the OUDA development team.
The stakeholders above hold administrative authority over the project and will serve to
guide its strategy and vision for implementation. Their role will be mostly informative.

32

The third (3) sub-unit is responsible for governing the OUDA digital library project itself,
and is headed by Gil Needham, its associate director. Responsible to her is the Digital
Libraries Programme Manager, Richard Nurse (RN), and the Library Services Manager,
Liz Mallett (LM). His team is responsible for systems, services, and infrastructure
development, hers for content provisioning, preservation management, and metadata
development.
A diagram (see Figure 4) of this organization can be seen below:

Figure
4
The project team is responsible for the day-to-day implementation of the OUDA digital
repository. The overall rationale for staffing OUDA is that the service should be pitched at
a sustainable level and should be based, as far as is possible, around existing levels of
staff. In most cases these can be accommodated by small changes in roles and job
description. We recommend that OUDA staff be made up of the following individual roles
and responsibilities.
Role Responsibility & Skills Required Responsible
to
Resource
time on
OUDA
Existing role
OUDA Project
Manager (AG)
Project management. Stakeholder,
communication and risk planning.
Deliverable and GANTT
scheduling. Budget and financing.
Reporting.
RN 0.5 FTE Project
manager for a
retired OU
project.
Open University
Archivist (RC)
Preservation strategy and policy.
Content and material provisioning
and supply. Digitization initiatives.
Archival expertise.
LM 0.5 FTE University
Archivist
Archive and
digitization
LM & RC 0.25 FTE (x2) Archive
assistants
Sponsor
NW
(Library Director)
Project
Governance
Library Leadership
Team

Accountable
Executive
GN
(Associate Director)
Reference
Group
External Stakeholders
(OU)
OUDA
Programme

Manager
RN
Project
Team

(Project
Manager:
AG)

Steering
Group
Internal Library
Stakeholders
OUDA
Content

Manager
LM

33

assistants (x2)
Systems
Administrator
Responsible for operations of
OUDA infrastructure and providing
basic frontline server/IT support.
RN 0.1 FTE IT Services.
Current
systems
librarian.
OUDA Digital
Repository Web
Developer (JA)
Infrastructure lead. Technical and
web development. Linked data
implementation. Interface
development. Data ingestion.
RN 1.0 FTE Currently a
temporary
post, to be
permanent.
Content and
Access Manager
(KB)
User experience/needs testing.
Content accessibility. Usability &
usefulness. Front-end. Promotion
and advocacy.
RN 0.5 FTE Currently a
digital projects
officer
Metadata
Development
Manager (LW)
Metadata standards and profiling.
Linked data. Classification and
vocabulary standards.
LM .5 FTE University
Metadata
Manager
Metadata
Development
Officer (technical)
Metadata standards and profiling.
Technical liason to Fedora
developer. Technical
documentation. Data
transformations and linked data
implementation.
LM .8 FTE Must create
role.
Advocacy,
promotion
Promoting the use of the digital
library. This role will fall across
various roles above (RN, LM &
KB), and will include OU
GN -- TBS
Action plan and timeline
Phase Scope Activities/Tasks Milestones Duration
(2 years)
1a Project
definition,
scope and
mandate.
Develop governance model
(above), project Steering
Group, and project teams.
Scope documentation.
Steering Group established. Mandate
developed. Project mandate approved
by project sponsor (NW).
2 months
1b Project
planning
High level detailed
planning. Milestones,
GANTT, risk management,
communications
strategy/plan, stakeholder
engagement
Documents:
--Scope
--Risk Strategy
--Communications Plan
--Stakeholder Engagement
Project team assembled
4 months
1c Work
packages
Stage 1:
Project team will begin
development across four
(4) work packages:
WP1—Fedora installed/configured for
digital content type bulk ingestion. Zend
Framework.
8 months

34

--WP1 (infrastructure)
--WP2 (content)
--WP3 (services)
--WP4 (standards/policies)
WP2—Identification, digitization, and
rights clearance of pilot content
WP3—User requirements, stakeholder
feedback assessed, services planning
and costing.
WP4—drawing up preservation policies,
and metadata profiles for digital objects.
1d Work
packages
Project team will begin
integration of work
packages and quality
control related to
stakeholder feedback and
informal assessments from
Steering Group and other
pilot users.
Fedora has ingested pilot content
(WP1), organized according to content-
types and course modules (WP2), and
must be preservation (TRAC) enabled
and metadata conforming (WP4). Front-
end (WP1) must expose content
according to service model expectations
(WP3) relevant to content types (WP2)
and metadata standards (WP4). Linked
data (WP4), front-end usability (WP3),
and rights and permissions clearance
(WP2).
8 months
Promotion,
user
feedback,
and
advocacy
Project advocacy and
feedback reports with
external stakeholder
groups.
Completed stages of communications
plan for promotion and feedback.
Using feedback from user groups for
Iterative quality control.
Front-end planning (WP1, WP3)
completed based on usability/usefulness
feedback.
1e Phase 1
closure
Assessment and project
(Phase 1) closure
Final (Phase 1) assessments complete.
Achievement of the desired end-state
(as per our planned goals and outcome
statements)
Lessons learned.
Stakeholder feedback and promotion.
Soft launch of OUDA prototype (to
specified stakeholder groups only)
2 months
Preliminary communications plan
Theme Objective Key message Relevant
Stakeholder
Digital
Preservation
We are preserving
content to ensure
legal compliance,
Protects investment as OUDA will enable re-
use and non-duplication of existing assets.
LTS, PVCs, Deans
and Assistant Deans,
library staff.

35

business continuity
and maintenance of
scholarly record.
Ensures asset authenticity, accuracy, and
completeness.
Ensures business continuity and helps
identify long-term trends
Ensures scholarly and cultural record
preserved as digital courseware objects
Open
Standards
Open software and
standards to lower
costs and increase
ROI
Open standards lower costs and increase
returns on investment by promoting:
Inter-operability
Vender neutrality
Efficient use of existing resources
Greater automatation
Flexibility and modularity
Robustness, durability, and sustainability
More options to optimize
Lower manageable risk
Quality
Increases staff skills
IT, KMi, LTS
Content Initially, OUDA will
contain pilot
content
A selection of digital and digitised archival
content.
Legacy course materials and their
component courseware items (videos, audio,
books, PDFs, images, etc.)
Various historical collections from OU
All stakeholders
Services Easy and open
internal
accessibility
We will develop a suite of digital library
services that will allow for OU staff and
administrators to access the repository.
All stakeholders
Linked Data Combing data
across silos
OUDA will employ RDF and linked data
technology to ensure the content can be
linked to and from internal and external
systems (through a SPARQL endpoint).
KMi, Comms (online
services), library
staff, OMU
Technical
Architecture
Flexible and low
cost system that
enables large scale
preservation
activities
Fedora Commons is open source repository
software for managing, preserving, and
linking digital content. It is flexible, modular,
scales to millions of objects, provides RDF
search, has RESTful APIs, and disaster
recovery utilities.
IT, KMi
Videofinder OUDA will provide
a sustainable
replacement for
Videofinder, a remit
of OMU
Videofinder is a non-sustainable system to
hold selected BBC content. With OUDA, the
BBC content will be preserved and related as
courseware to the module records it was
originally produced for. It will expose this OU
content to the world and support informal and
formal learning opportunities from it.
OMU
Documentum OUDA will
complement
Documentum.
OUDA is not competing with Documentum.
Documentum is the OU’s document
management service.
LTS, Rights and all
stakeholders

36

Forecast budget for OUDA
Staff Expenditures
Staff Member Days per
month
(approx.)
£ Monthly
(approx.)
# Months
active
Phase
active
£ Total
(approx.)
OUDA Programme Manager
(RN)
4 800 24 1a-1e 19,200
OUDA Content Manager (LW) 4 800 24 1a-1e 19,200
OUDA Project Manager (AG) 7 1,500 24 1a-1e 36,000
Open University Archivist
(RC)
7 1,500 24 1a-1e 36,000
Archive / digitization
assistants (x2)
7 (3.5 x 2) 750 18 1c-1e 13,500
Systems Administrator 1 200 18 1c-1e 3,600
OUDA Digital Repository Web
Developer (JA)
18 3,500 18 1c-1e 63,000
Content and Access Manager
(KB)
7 1,500 18 1c-1e 27,000
Metadata Development
Manager (LW)
7 1,500 18 1c-1e 27,000
Metadata Development
Officer (technical)
7 1,000 18 1c-1e 36,000
Advocacy, promotion 4 (average) 500 18 1c-1e 9,000
Sub-Totals 13,550 £ 271,500
Non-Staff Expenditures
Item £ Cost
Digitization, rights assessment, and clearance costs 18,000
Front-end designer (sub-contracted) 5,000
IT Computer equipment costs 45,000
Advocacy and promotional events/materials 1,000
DOI (Digital object identifier) costs 3,000
Sub-Totals £ 72,000
Grand Totals (approx.) £ 343,500

37

Pilot testing and scalability
There will not be any pilot testing of OUDA per se. Given that this is in the first instance a
software development project, we will use an iterative and incremental approach to pilot
testing the software evolution. Team members work in close proximity (the same OU
Library Services building) and will engage in constant cross-functional, self-organizing and
adaptive meetings and scheduling. All team members will participate continuously in
software testing and quality control. There will be a minimum amount of usability testing for
web accessibility standards, and this will involve a lot of cross-browser functionality testing
and usable interface design prototyping. It will also involve ensuring content and metadata
is exposed at the right levels and ensuring its discoverability via facets, well-placed blocks,
breadcrumbs for re-traceability, precision granularity of search functions, and through
exposing related content (e.g., OpenLearn) through linked data similarity.
As part and process of the marketing plan (see below), we will find natural breakpoints to
“pilot test” promotional prototypes of its forecast look, feel, form and function. During
Phase 1, this will unlikely involve interaction with the software system itself. Instead, it will
involve drawing up posters, presentations, and mock-up prototyping based on its evolving
mission. These prototypes will serve simultaneously as promotional material as they will
material from which feedback from relevant stakeholder groups will be collected and
assessed. This, as described in the main literature review, will be iterative, and will sustain
and increases stakeholder attention when they witness proposed amendments being
integrated into follow-up prototyping sessions.
There are very few issues of scalability with regards to project size, except insofar as
technical infrastructure scalability is concerned. Fedora scales to millions of digital objects,
and the OU Information Technology unit can easily accommodate OUDA on load balanced
and mirrored servers with many thousands of terabyte capacity.
OUDA itself is itself a prototype that will only be launched “softly” to a select group of
internal stakeholders after Phase One development is complete. For the most part, its
development can only be scaled up. Its scope specifies a minimum level of infrastructure
and resources for its development. Its Phase One development will include a minimum
level of course module (three, out of hundreds), but for completion must include various
kinds of materials (video, audio, books etc.) and its associated metadata. The plan is that
OUDA will be scaled up to also include non-course material such as digital library image
collections.

38

SWOT analysis
Strengths Opportunities
• The remit of the OU Archive is to
‘preserve’ institutional memory.
• There is clear demand from OU units
for archiving study materials.
• Digital repositories are a consistent
feature of most university library
services.
• The Fedora is acknowledged as a
leading application system.
• The system will comply with the Open
Archives Initiative Standards (OAIS).
• Library Services has started to build
clear expertise in the area.
• Our developer has expertise in digital
libraries and video material.
• Very strong metadata expertise and
acknowledged leadership.
• Good links with linked data experts in
KMi.
• Strong links established with both
digital curation and preservation
communities.
• No other unit has a remit to preserve
materials to national/international
preservation and metadata standards.
• There is clear demand and a need as
units are becoming aware that content
is being lost.
• Publicity of OUDA given STELLAR
project has already raised importance
to and interest in archiving legacy
course material.
• Faculties are enthusiastic about
archiving their content (cf. STELLAR).
• Both the VC and PVC have heard
descriptions and have seen the
potential, and are interested.
• OUDA Phase 1 success will provide a
platform to bid for additional
resources to carry out more
digitisation work. This may help the
library grow and sustain itself as an
important resource.
Weaknesses Threats
• There is more demand than we can ever
satisfy, as resources are limited.
• We do not have strong support from
corporate IT (there is some opposition).
• Our technical resources and technical
expertise are limited to a few individuals
who may or may not have fixed term
contracts in place.
• Library Services Unit is being asked to cut
back on resource budgeting, calling into
question
•
• IT believe that a single system should
be used to store OU assets
• LTS e-production objectives might
potentially conflict with OUDL
objectives
• That some elements of the archival
content are still ‘live’ for example for
BDU and OMU. This could create
questions of legacy content
duplication and when and if these
resources will be deprecated in favour
of OUDA.
• IT may not support KMi linked data in
the long-run.
Limitations and assumptions
This plan assumes first and foremost that the current levels of resource funding within the
library will be maintained over at least the next two years. It assumes that the project will
receive high priority status within OU Library Service such that staff resource time can be

39

re-allocated towards specifications of the action plan outlined above. This project plan
assumes that courseware content across the OU will be available to the Information
Management teams described above, and that no particular academic or production unit
will protest (i.e., on legal or copyright grounds) these resources being archived within the
remit of OU’s Library Services. We make the assumption that the library is responsible for
engaging with digital preservation projects and also has the expertise to carry out the work
identified in this set of project plans.
Our main limitation is the set of human resources currently available within OU’s Library
Services. Without a professional archivist, metadata developers, and digital library
systems developer, this project would be impossibly constrained. These functions are
essential. The library has sufficient resources to handle content and accessibility services
as well as project management functions.
Another limitation is IT acceptance to host the server infrastructure upon which these
services will depend. This may require extra costs as involved in intra-university sub-
contracting policy, especially given that OU Library IT services do not have enough
capacity to serve as a full solution. OU Library IT services, as explained above, will serve
an informative role. It can provide short-term developmental servers (and will). But it
cannot provide large-scale, long-term, load balanced and mirrored systems, with full
backup and disaster recovery plans in place, nor the technical resources to maintain
these.
The use of linked data also depends fully on the cooperation of KMi’s linked data SPARQL
endpoint, which has evolved a lot over the years and now represents as URIs all of the
courses and the digital objects making them up (see open.data.ac.uk). There is also some
discussion as to whether OU Central IT will support the adventures of KMi, as KMi is a
semi-autonomous research institution within the OU supported by many independent
sources of funding external to the OU’s operating budgets.
Reflection
Planning OUDA has been an excellent exercise in balancing the attention to technical
requirements with those of its usually non-technical set of stakeholders and potential user
groups. The project itself focuses on developing a strong suite of software supporting a
preservation system. Preservation itself is a worthy goal. But the system must fit into the
day-to-day workflow processes of library services, and for this reason making a case for its
development based merely on an analysis of library user needs is difficult enough. The
library fits within university-wide processes and is also subject to its overarching strategic
goals.
For this reason it is also important that library function complements the function of its
sister units (OMU, LTS, KMi, the academic units, and the staff, faculty and course
producers making these up), as well as its managing units keenly interesting in both
enhancing the reputation of the OU, but also in the business realities of trimming budgets,
increasing returns on investment, and creating a institution that is poised to survive
historical transitions into the digital age and out-perform its competitors in terms of
educational value to its customer base.

40

This was a difficult balance to make. Planning documentation for the outcomes was
especially demanding. It was tempting to only look at long-term outcomes (4+ years), but
that would be premature. It is the long-term outcomes that will directly address ROI,
production efficiency, and OU reputational enhancement. But for this phase we must look
at the intermediary goals that within two years our small set of resources are actually
capable of producing, such that we at least can convince higher-level stakeholders that is
continued support is justified and on its way to delivering on the longer-term outcomes that
its development is meant to serve.

Marketing OUDA preservation capabilities to internal stakeholders
Introduction
University’s online educational model and corporate e-business. This Phase One
developmental prototype of a digital repository is in response to increasing demand from
Open University stakeholders to provision content management and preservation services
for university generated material.
The OU Digital Archive (OUDA) is a university-funded project. It is the result of a few years
of informal conversation between various university units, the most important of which
include the Pro-Vice Chancellor’s office for Learning and Teaching (PVC-LTQ), as well as
the Director of Library Services and the Associate Director of Information Management
and Innovation (see the stakeholders analysis in the planning documentation).
Existing marketing efforts for OUDA include various actions taken on stakeholder groups
defined in the previous planning documentation. This includes as high priority maintaining
PVC-LTQ interest in the project insofar as OUDA is in its capacity able to satisfy the
strategic goals of increasing learning and teaching efficiency, developing pedagogy,
enabling scholarship, and increasing business efficiency.
The first and most important of these marketing efforts comes as a result of the JISC
funded STELLAR project (non-published, 2013) run by OU Library Services. When
stakeholders across the university spectrum (academic, staff, senior administrators, asset
managers, content producers) were surveyed, their responses showed high levels of
stakeholder interest and recognised need for the provisioning of preservation services for
non-current learning material. This need included mention that easy access to non-current
learning material would inspire academics and content producers when producing new
content; and also that it would serve university strategic value by eliminating unnecessary
duplication by encouraging course material re-use.
Relevant literature
There are many reasons for not implementing a digital repository. A repository involves
high costs to changing workflows and university processes. It can also be very expensive
to deploy and maintain the infrastructure necessary, not to mention the high learning curve
of those who will be involved in its use. Also, there are copyright concerns, definitions for
content inclusion/exclusion, issues of organisational adherence, as well as those of

42

content duplication and its integration with other existing systems (see Davis & Connolly,
2007).
In the planning documentation above, the benefits outlined and the impact these would
have on library and university need were outstandingly in favour of implementation. For
this reason, we must develop a framework under which we can promote OUDA as a
product, albeit an intangible product (see Ferreira, Rodrigues, Baptista & Saraiva, 2008),
that will add value far outweighing the costs involved in its implementation. The most
relevant form of marketing during the Phase One developmental prototyping of OUDA will
involve managing this stakeholder perception. In so doing, we will closely assume the
work of Heleen Gierveld (2006).
Gierveld stresses two of the “8 Ps” of the service marketing mix (Wirtz, Chew & Lovelock,
2012), product and promotion. She goes on to argue that that the development and
management of a digital repository depends on strategic social marketing tools that
communicate by informing and explaining, educating and stimulating, involving and
inviting, and attracting the attention of stakeholders and potential users.
Gierveld outlines a four-strand communication strategy that we adapt here. It includes the
two-sided process of university stakeholder consultation (1) in order to cultivate strong
relationships and learn about and adapt to their needs during developmental prototyping.
In so doing, we must segment their domains of expertise and organizational purpose and
survey the potential value having a repository of legacy material available might add to
their workflow and digital lifecycle processes. This involves the perception of repository
trust (Steinhart, Dietrich & Green, 2009; Ross, 2012; Prieto, 2009), the main goal of the
TRAC audit outlined above. Data sharing hinges on trust in the authority, authenticity,
accuracy and credibility of data (Van House, 2002).
The consultation process itself will provide the fodder for planting the seeds of product
promotion. That is, the consultation process for information infrastructure projects is
iterative and will occur over the lifespan of the project (Schwalbe, 2007). Thus, Gierveld’s
second order of business in creating a successful repository will be to properly profile (2)
the potential service. Wary of the language we pitch our efforts given various stakeholder
technical understanding, we must profile—in the form of a talk, poster, presentations,
system mock-ups, wireframes, or other such media—our plans for developing the
infrastructure, and how a system of trustworthy preservation and long-term access will
impact (i.e., add value to) each stakeholder’s set of interests given their foreseeable need
and usage of the repository.
Profiling the iterative improvements in system capabilities will serve to then attract or pull
(3) the relevant stakeholders into participating more fully in its implementation. That is,
good profiling will inform users of benefits it will provide their segment and other
practicalities of its eventual use. Also, witnessing through effective promotional profiling
that earlier consultative feedback has been integrated will thus serve to encourage and
push (4) their continued allegiance to the product. That is, it will encourage conditions that
involve the continued participation of relevant stakeholders.
Marketing goals and outcomes
According to our literature review, profiling the developmental prototypes serves a three-
way purpose. It creates discussion and interest in their continued support (pull). It also

43

provides a way to illustrate earlier feedback has been integrated, thus reinforcing (push)
the belief that their continued participation and feedback is important. Additionally, it
provides a forum for continuous evaluation and more consultation.
This feedback loop represents the overarching goal of our marketing strategy. In other
words, the work of marketing will be to inform relevant stakeholders of repository planning
initiatives. Each initiative will include materials that profile the proposed service. Thus, our
first concrete goal is:
Goal 1: To inform stakeholders with materials profiling developmental stages of how
the repository prototype OUDA will impact their segment.
The outcome of this goal will be the generation of stakeholder feedback regarding these
developmental proposals. In this sense, feedback will also fulfil the purpose of engineering
requirements into our software development planning. While constrained first and foremost
through the requirements of developing a preservation platform, its development is flexible
with stakeholder need/interest regarding issues of content selection, content organisation,
its usability and usefulness, discoverability and search mechanisms, and integration with
other existing software platforms.
Adapting these requirements into our planning will constitute the second goal of our
marketing strategy. That is, our second concrete goal is:
Goal 2: To integrate requirements feedback into existing software development,
and effectively profile these changes and updates in marketing materials in a way
that reinforces stakeholder importance, and cultivates continued interest.
The outcome of this goal will be continued interest in the project in ways that generate
excitement over the perceived value of the product. The product must be perceived to
impact upon one or more of the interests or needs of that stakeholder segment. This might
involve ease of discovering non-current course module assets for re-use when producing
new courses. Or, it will showcase how linked data between assets on different production
systems will operate (e.g., BBC and OpenLearn). It also might involve adding value to
university reputation as historical course modules can be accessed and browsed by
potential students or researchers. If feedback has not been integrated, stakeholders will
lose interest, as their interests will be seen as not having importance.
The third goal of our marketing strategy will be the following:
Goal 3: To effectively promote OUDA as a functioning preservation system and web
portal that enables easy access to non-current course modules in a way that
facilitates the eventual re-use of the existing assets (i.e., video, audio, image, PDF,
texts) that make up each course module, as well as enhances the image of the
Open University as one of the most innovative online universities worldwide.
The outcome of this goal will be satisfied if continued funding can be secured. A
functioning and internal soft release of an OUDA prototype for internal use will serve as
the basis for promoting the continued support and funding of its cause. That is, OUDA at
this stage will only provision a minimal amount of content for a few course modules. Most
notably, the first module ever created (S100, Introduction to Science), the module that
became a major impetus for Open University success, will be included. Given the young
age of the university, many influential stakeholders and faculty are still very familiar with

44

the production and success of this module, and have every reason that its preservation
should be secured.
Positioning statement
OUDA aims to enhance the visibility and academic reputation of the Open University (OU)
by preserving selected Open University historical, learning, teaching, and research
content. Exposing this learning and research material will demonstrate the quality and
increase the usefulness of OU’s pedagogical methods and illustrate how they have
developed over time.
Because these resources have been expensive to produce, there is a growing demand
from faculty, academic researchers, university content producers, and library archival
services to address the long-term sustainable management of this growing corpus of
legacy material. For this reason, OUDA will operate as an important repository for the
long-term preservation of OU digital resources.
The availability of these resources to wider audiences will enhance the reputation of the
OU’s role as a leading innovator for quality online education. Their preservation will also
facilitate university content e-production by increasing digital asset re-use through easy
discovery and access of relevant digital material. Re-use and non-duplication will lead to a
greater return on investment, saving on production costs, and delivering long-term
institutional economic viability and competitiveness.
Target audience and key messages
Target audience
The planning documentation above outlined key internal (library) and external (university
and public) stakeholders. The development of OUDA will impact the potential interest of
these stakeholder segments in various respects, depending on their needs. These also
were described in the planning section above. For this reason, when marketing the value
of OUDA, careful targeting to these distinct segments given their distinct needs and
interest in OUDA will be an essential element to its continued support and university-wide
integration. Below, we draw attention again to these various stakeholder segments.
Library Stakeholders University Stakeholders
Library Director
Nicky Whitsed (NW)
Pro-Vice Chancellor of Learning and Teaching
(PVC-LTQ)
Professor Belinda Tynan
Associate Director (Information Management and
Innovation)
Gill Needham (GN)
LTS (Learning and Teaching Services)
OUDA Programme Manager (RN) OMU (The Open Media Unit)

45

OUDA Project Manager (LM) KMi (Knowledge Media Institute)
OUDA Systems Developer (JA) OU Central Academic Units
OU Archivist (RC) OU Information Technology
OU Metadata Development Manager (LW) External Stakeholders
Library Academic and Student Support Leader OU Students
Library IT team Wider HE and library community (academics and
staff/students from other institutions)
Library IM team Linked data community (when data is exposed in
RDF format)
Other library staff Digital preservation an curation communities
General public
Key messages
With long-term preservation the goal, OUDA is being developed in alignment with the OU’s
mission and core value statements outlined in the planning documentation. That is, the
repository should create value insofar as it addresses the four OU Focus Statements
above. That is, it should 1) make teaching and learning more efficient, 2) innovate
pedagogy to increase OU’s reputation, 3) promote OU’s research impact, and 3) increase
business efficiency in the production of course materials (and hence decrease costs).
Alongside our positioning statement above (which, in most respects, iterates these
strategic goals), these statements will provide the general orientation our key marketing
messages to high-level university stakeholder groups will take when promoting OUDA as
the university-wide answer to long-term course material preservation.
More specific objectives driving the development of OUDA are outlined below. Beside
each objective we list the key message and its relevant stakeholder group to whom this
key message will be communicated at various intervals over Phase 1 development. It is
worth mentioning here that this forms a rudimentary basis of a communication plan. During
Phase 1b (project planning), a far more precise communications plan for the purposes of
marketing and promoting OUDA to individual stakeholders within these segments will be
developed.
Many of these key messages must be tailored to mitigate many of the perceived project
risks we foresee stakeholder groups as having. For instance, the external group of
stakeholders, with the exception of PVC-LTQ, may potentially view OUDA as a threat to
their existing services. Care must be taken to convince these groups that OUDA will seek
to complement their existing digital architectures and services, not replace or disrupt them.
Another risk to mitigate is that OUDA may complicate the workflow processes of other
units. The risk that OUDA is not adopted wholesale but instead becomes just another
digital object storage sile, amongst many others, is real.

46

Theme Objective Key message Relevant
Stakeholder
Digital
Preservation
We are preserving
content to ensure
legal compliance,
business continuity
and maintenance of
scholarly record.
Protects investment as OUDA will enable re-
use and non-duplication of existing assets.
Ensures asset authenticity, accuracy, and
completeness.
Ensures business continuity and helps
identify long-term trends
Ensures scholarly and cultural record
preserved as digital courseware objects
LTS, PVCs, Deans
and Assistant Deans,
library staff.
Open
Standards
Open software and
standards to lower
costs and increase
ROI
Open standards lower costs and increase
returns on investment by promoting:
Inter-operability
Vender neutrality
Efficient use of existing resources
Greater automatation
Flexibility and modularity
Robustness, durability, and sustainability
More options to optimize
Lower manageable risk
Quality
Increases staff skills
IT, KMi, LTS
Content Initially, OUDA will
contain pilot
content
A selection of digital and digitised archival
content.
Legacy course materials and their
component courseware items (videos, audio,
books, PDFs, images, etc.)
Various historical collections from OU
All stakeholders
Services Easy and open
internal
accessibility
We will develop a suite of digital library
services that will allow for OU staff and
administrators to access the repository.
All stakeholders
Linked Data Combing data
across silos
OUDA will employ RDF and linked data
technology to ensure the content can be
linked to and from internal and external
systems (through a SPARQL endpoint).
KMi, Comms (online
services), library
staff, OMU
Technical
Architecture
Flexible and low
cost system that
enables large scale
preservation
activities
Fedora Commons is open source repository
software for managing, preserving, and
linking digital content. It is flexible, modular,
scales to millions of objects, provides RDF
search, has RESTful APIs, and disaster
recovery utilities.
IT, KMi
Videofinder OUDA will provide
a sustainable
replacement for
Videofinder, a remit
of OMU
Videofinder is a non-sustainable system to
hold selected BBC content. With OUDA, the
BBC content will be preserved and related as
courseware to the module records it was
originally produced for. It will expose this OU
content to the world and support informal and
formal learning opportunities from it.
OMU

47

Documentum OUDA will
complement
Documentum.
OUDA is not competing with Documentum.
Documentum is the OU’s document
management service.
LTS, Rights and all
stakeholders
Message delivery strategies
Tools, timeline, and responsible parties
Given OUDA is in its infancy, and was commissioned under the remit of OU Library
Services, the most important first step to marketing its university-wide viability must include
high-level stakeholder meetings. Initially, these meetings will involve carrying forth the
mandate approved at the highest level of OU Library Services, including the project
sponsor (Nicky Whitsed, Library Director), the OUDA Steering Group, and the Library
Leadership Team (see the library organizational diagram in the planning documentation).
That is, after the mandate and project scope has been approved (Phase 1a), university
stakeholder leaders will be briefed on the project at monthly meetings. These briefings will
involve formal meetings between the OUDA Steering Group and corresponding senior
managers across university stakeholder segments listed above.
Below the initial timeline for Phase 1 is reproduced, along with the key tools and their
nature of delivery (i.e., the medium and those to receive the message). This very general
timeline corresponds to the action plan and timeline developed in the planning
documentation.
Phase Scope Duration
(Out of 2
years)
Marketing Tools Nature of delivery Time
Span of
delivery
1a Project
definition,
scope
and
mandate.
2 months Meetings. Briefings. Email
correspondence. These
briefings will outline key
strategic goals and
outcomes of the service.
Tools will mostly involve
one-page briefs, short
presentations, and signed
agreements for intra-
university cooperation.
Monthly meetings will be
established between relevant
OUDA Steering Group
members and relevant
university stakeholder groups.
Only high-level managers.
Length of
project
Phase
1a – 1e
Responsible Parties OUDA Steering Group with high-level managers from PVC-
LTQ, OMU, LTS, KMi, OU Academics, and OU IT Services.

The OUDA Steering Group includes the OUDA Programme Manager (RN) and the OUDA
Content Manager (LM), who manage the Project Manager (AG) and the Project Team.
This second level of management will work with both external and internal (i.e., other
library) stakeholders in order to promote and generate feedback regarding the actual

48

details of project implementation and how it will or will not impact upon services carried out
within and between these departmental (i.e., OU Unit) processes.
As specified on the action plan timeline, the first four months will involve meshing out the
details of project management. Thus, as RN and AG also belong to the OUDA Steering
Group, higher-level strategic directives will trickle down into project planning
implementation. That is, issues of project scope, risk management, critical paths of service
delivery, and the finer details of communications will be set forth. These documents will
serve as internal promotional material to market OUDA to internal library stakeholders and
gain their feedback and general acceptance. This includes feedback from those internal
stakeholders who will form the basis of the project team.
After the project management phase (1b), RN and AG will be responsible for maintaining
feedback channels open between various library managers with a stake in OUDA. This
includes library staff responsible for issues of licensing, content provisioning, academic
and student services, and business and IT systems. More importantly, it will involve
coordinating the efforts of the project team itself and their needs and interests relating to
the project (i.e., metadata, digitization, archiving and preservation, repository
development, and content and access services). RN and AG will also serve to facilitate
meetings and briefings between project team members and identified external
stakeholders who will be most affected by and will most affect specific and granular details
of software implementation (e.g., in course production, academic use, digital content re-
use, linking metadata, and IT server infrastructure).

(Out of 2
years)
Span of
delivery
1b Project
planning
4 months Meetings and briefings.
Email correspondence. Will
advise on and utilise the
tools developed by the
project team (see below).
Will facilitate and help
coordinate connections
between relevant external and
internal actors with relevant
active stakes in OUDA with
implementation actions of
project team. Will advise on
the creation of and sometimes
represent marketing and
promotional material
developed by the project team.
Promotional material will
inform and generate feedback
for continued prototyping.
Phase
1b – 1e
Relevant Parties OUDA Programme Manager (RN) and OUDA Content
Manager (LM). Internal library staff, including project team
(metadata, digitization, archiving and preservation, repository
development, and content and access). External actors, such
as course producers (OU, LTS), KMi, IT Services, academic
users.

49

Our use of marketing materials, as informed by our analysis of the marketing literature
above, will thereby serve a threefold purpose. Quoted from above, promotional material
will:
Create discussion and interest in their [relevant stakeholder] continued support
(pull). It also provides a way to illustrate earlier feedback has been integrated, thus
reinforcing (push) the belief that their continued participation and feedback is
important. Additionally, it provides a forum for continuous evaluation and more
consultation.
The following diagram (see Figure 5) is an abstract characterization of how this feedback
loop will function:

Figure
5
The diagram below (see Figure 6) outlines how this promotional feedback loop will function
within the context of OUDA.
The black arrows (à) indicate the direction in which specifications and need requests are
passed down from stakeholders to library teams. These specifications will be generated
from initial stakeholder briefings, during which OUDA will first be promoted. The red dotted
arrows (ß) indicate the direction in which planning documentation, step-wise software
prototyping specifications, and posters and talks are used to market and promote
continued interest and further stakeholder engagement.

50

Figure
6

The project team itself will be responsible for the creation and often the delivery of
promotional products. Each member of the team has expertise in various areas, as
demarcated in the planning documentation. Much of our marketing plan at this level must
focus on promoting and gaining feedback about often quite technical matters (i.e., sharing
metadata and archiving content across university units; archiving modules and their textual
and audio-visual content stored on various servers; developing OUDA packages that help
not hinder course material re-use and module creation; preservation).
That is, we must show (i.e., market or promote) OUDA is being developed in a way that
can and does adapt to user needs and will deliver on its stated strategic goals in concrete
ways in order that it will eventually add value to its stakeholder base of future users. Each
team expert will continue networking across his or her sphere of university influence, and
will also be guided to extend this sphere as recommended by Programme and Content
Managers.
In Phase 1c outlined below, this will mostly involve email, talks, and briefings in order to
generate discussions regarding concrete need. In the second year, after the repository has
some semblance of shape and form, more traditional forms of marketing materials will be
created. This will include posters, postcards, presentations that highlight the planned look
and feel of the repository website; it will also include sessions (one-on-one or in groups)
with stakeholders to prototype and gain feedback from the ways in which users will interact
with the content, ensuring that we maximise its usefulness to various kinds of users.
Additionally, OUDA will also be the subject of its own university blog that outlines its
development over time. All members of the project team, as well as the Programme and
Content Managers, will be responsible for updating the blog on a bi-weekly basis. The
blog will serve as the official running record of OUDA developmental milestones.
External
mid-‐level
stakeholders
(e.g.,

course
producers,
academic
users,

data
linking,
content
providers)
External
stakeholder

management
OUDA
Steering
Group
Project
Team
Other
Library
Staff
Programme
(RN)
and
Content

(LM)
Managers

51

(Out of 2
years)
Span of
delivery
1c Work
packages
Stage 1
8 months Meetings, email and
briefings to develop
conversations regarding
specifications and
requirements based on the
expertise of the project
team.
OUDA blog.
Continual weekly meetings
between project team
members and their relevant
university unit cohorts. That is,
archivists must collect content;
metadata experts must share
data; repository developer
must specify architecture with
IT; archivist must develop
preservation and licensing
policy with course producers;
content and access manager
must organise content
according to academic usage
and course module production
workflow, etc.
Phase
1c – 1e
Relevant Parties Project teams, advised by Programme and Content
Managers, with their respective university unit partners
1d Work
packages
Stage 2
8 months Continue 1c marketing
tools above, and…
Posters, postcards,
presentations (e.g.,
PowerPoint), project
wireframes, repository
infrastructure preservation
processes (ingestion
workflows), and content
usefulness prototyping
sessions.
OUDA blog.
Continue 1c marketing delivery
above, and…
University events (e.g., “Learn
About Fair”, Open Library
Day), invited talks and
presentations. Poster sessions
with official project briefings
and demonstrations. Hand-
outs and flyers. Continued
front-end prototyping.
Phase
1d – 1e
Relevant Parties Project teams, advised by Programme and Content
Managers, with their respective university unit partners
The final stage (1e) of OUDA Phase 1 development involves its final assessment and
official internal “soft launch.” This launch will be the most significant form of promotion. As
a beta product, various university stakeholders will be encouraged to engage in its use
and provide feedback. As mentioned, it will only contain minimal content (i.e., three
complete course modules, including the OU’s first course ever, S100). It will also include
semantic links to other “similar” university generated content (BBC videos, iTunes
University, OpenLearn, etc) through the use of semantic metadata.
The library will invite all significant stakeholders and the university public in general to this
launch event held at the library, during which there will be a talk, posters, and live on-

52

screen demonstrations. OUDA staff will also be prepared to answer questions and discuss
its capabilities and its future developmental path.
(Out of 2
years)
Span of
delivery
1e Phase 1
closure
2 months Final posters, flyers, and
postcards.
Beta OUDA repository
website launched for
internal use. Will include at
least 3 course modules
(including S100) and all of
the kinds of content that
course contained (texts,
audio, video, books,
subtitles, and images).
OUDA blog.
Final assessment phase.
Questionnaires, surveys
etc. (see assessment
documentation).
Some promotional material will
describe OUDA and its
purpose.
Other promotional material will
announce the “soft launch” of
the OUDA repository website
to university staff campus-
wide. Emails, university-wide
‘press’ releases, and posters
hung around campus will
announce the soft launch
lunchtime event that will take
place in the OU Library. This
will include OUDA
demonstrations, Q&A
sessions, and staff availability
for discussion and specific
questions.
Phase
1c – 1e
Relevant Parties Campus wide event. All relevant stakeholders, external and
internal, and all project staff. PVC and Library Director will
deliver address, and project team will demonstrate product.
Marketing costs
The marketing plan specified above will be managed by the OUDA Steering Group and
the Project Teams as described. However, the OU Library has its own in-house marketing
team responsible for university-wide branding and marketing policy. Thus, the OU Library
marketing teams must approve all marketing materials. The branding of the repository
must also conform to university standards regarding branding and accessibility. The
marketing team has inline budgeting established for this expenditure.
The forecast budget for OUDA was delineated within the planning documentation. It
included an inline item of £1000.00 for advocacy and promotional events and materials.
This will include any costs of printing and publishing. All other marketing costs fall within
the remit of the job responsibilities described for each member of staff.

53

Mockups of selected marketing methods
Posters
Open University Digital Archive
Explore the OU’s world-class heritage
ouda.open.ac.uk
OU STUDY MATERIALS HISTORICAL OU TV AND RADIO
BROADCAST PROGRAMMES
HISTORICAL OU IMAGES
Beta

54

Informational posters and handouts
Open University Digital Archive
Open University Archive
• Teaching materials archive – printed study materials, TV
programmes, radio programmes
• Historical images – people, places, events
• Historical material from OU people
• Student study experiences – e.g. Open Forum programmes,
oral history
• Research project papers
• Physical artefacts – e.g. home study kits
Remit of OU Archive is to
select and preserve material
from the OU’s history, covering
teaching materials, methods,
activities, people and places
OUDA Vision
“Provide stewardship and strategic advocacy for sustainable digital
preservation of teaching materials and research data management
services for long term access and reuse”
Library Services strategic priorities 2010-2014
“Plan and develop the OU Digital Library, to make
maximum use of existing and emerging internal,
sustainable collections to support both formal and
informal leaning” Library Services operational plan 2011-12
“Develop and implement a Digital Preservation Strategy
and associated services to ensure that the OU’s unique
content (from learning, teaching and research) is managed,
preserved and exploited effectively” Unit priorities 2013-2016

55

Project postcard / info-graphic
Open University
Digital Archive
The Open University Digital Archive
(OUDA) provides access to materials
from the OU University Archive, and will
help to preserve and manage the
materials.
OUDA includes study materials, videos,
audio recordings, images and other
materials from the University’s 45-year
history.
OUDA will launch in Beta to OU staff in
August 2015.
OUDA will facilitation course module
production and academic re-use of
previous course module content.
OUDA will launch with a limited
selection of content of different types.
More content will be added over time.
OUDA beta will include:
• A small selection of OU study materials,
including a complete digitised version of S100,
the first Science foundation course from 1971.
Includes books, video, audio, and
supplementary materials
• Over 160 OU historical images from the late
1960s onwards, including significant OU
events and people
• 30 Open Forum videos
• a sample archived VLE website from 2006
(S103)
• a selection of TV and radio broadcasts created
by the OU in conjunction with other
broadcasters including the BBC.
Contact archive@open.ac.uk for more information
!"#$%&$'(#)*'+,%-'.'+/0%1)23'(#
45"06)#%+3#%!&7*%86)09:20/**%3#)'+/.#
!"#$%!&'(%$)%"*
!&%;<&-=%>1<4?@1A; B@;<!?@C1A%!&%<D%1E-%?1-@!
F?!1-C1;<%G?!H?1>>4;
B@;<!?@C1A%!&%@>1H4;
+',$

56

Reflection
Developing a marketing plan for a repository development project has not been a
straightforward task. Firstly, the marketing plan is in many respects a communications plan
for developing and maintaining relationships with relevant stakeholder groups whose
needs must be satisfied if the project is to have success.
In this respect, the marketing plan was developed somewhat along the methodological
lines of software iteration and requirements engineering. That is, promoting OUDA as a
repository would take place alongside its software specification. When meeting with
relevant stakeholders, the plan is to promote working versions (conceptual, and later
actual working prototypes, wireframes, and content discovery processes) of OUDA as
step-wise proposals for action. These proposals will in effect operate as promotional
materials to market OUDA according to perceived stakeholder need. The meetings and
briefings designed to generate interest in OUDA will have the additional effect of creating
discussion and feedback at each stage of development. They are designed to reinforce
the notion that their continued interest in the project is essential, insofar as their needs and
requests have been incorporated into continued project prototyping.
This has required somewhat of an abstract conceptualization of timeline and action
planning. This was done deliberately given that these feedback loops cannot be properly
predicted. Software development in small teams must be agile and adaptive in nature. Our
plan is also in its inception, and its first two months will additionally include project
management planning (the OU applies PRINCE2 project management methodologies),
during which time more specific documentation of communication, stakeholders, cost, risk,
change control, work breakdown structures, and project scope will be developed, before
even which the project team will go to work. This PMA (planning, marketing, and
assessment) documentation forms an excellent first step towards this goal, and in a sense
operates as a project proposal specification.
A full-fledged communication plan as part of project management activities will include far
more specific means of promotion and outreach to not only general stakeholder segments
as we outline here, but the very individuals that must be targeted within each of these
segments, as well as how often and by which means they will be targeted.
Thus, I do believe that marketing plans for a software development project must also
remain flexible and agile, especially with such a small team. I have here provided a few
mock materials for how the repository might look, feel and operate. However, given the
abstract nature of our feedback-based plan, user need and interest might develop in any
direction, and the project should accommodate those needs while still remaining within the
remit of developing a repository whose overarching goal is for long-term preservation and
access.

Assessing OUDA as an accessible preservation system
Introduction
University’s online educational model and corporate e-business.
The vision of OUDA aims to enhance the visibility and academic reputation of the Open
University (OU) by preserving selected Open University historical, learning, teaching, and
research content. Exposing this learning and research material will demonstrate the quality
and increase the usefulness (i.e., through re-use) of OU’s pedagogical methods and
illustrate how they have developed over time. Because these resources have been
expensive to produce, there is demonstrated demand from faculty, academic researchers,
university content producers, and library archival services to address the long-term
sustainable management of this growing corpus of legacy material.
It is these stakeholders that represent in the first instance our user base. In the planning
documentation, we analysed the needs from which these stakeholder user groups can
benefit given the development of OUDA. In the documentation that follows, it is our
intention to set out plans to formally assess the impact the service has on addressing
these needs. In the documentation that follows, we outline a model of assessing Phase
One of the development and deployment of OUDA as a digital repository software system
for preservation and course module re-use.
OUDA is in Phase One of planning. It is a two-year plan. This first phase describes the
implementation details for a developmental prototype of a digital repository to archive and
preserve university generated content. Preservation itself is a worthy goal. But the system
must also fit into the day-to-day workflow processes of library services. The library
functions within university-wide processes and is subject to its overarching strategic goals.
For this reason it is important that library function complements the function of its sister
units (OMU, LTS, KMi, the academic units, and the staff, faculty and course producers
making these up). But that it is also created to fulfil higher-level management goals of
enhancing OU reputation, trimming budgets, increasing returns on investment, and
creating a institution that is poised to survive historical transitions into the digital age and
out-perform its competitors in terms of educational value to its customer base.

58

It is tempting to only look at long-term outcomes (4+ years), but that would be premature.
It is the long-term outcomes that will directly address ROI, production efficiency, and OU
reputational enhancement. In order to put the assessment of Phase One into longer-term
context, however, it is worth outlining the long-term outcomes of OUDA. Long-term
outcomes are as follows:
• The OU Archive is able to use the OUDA as their main cataloguing environment for
study materials.
• OUDA is the main interface for digital preservation workflows and processes.
• Library management is able to use usage reports generated by OUDA to make
informed business decisions.
• Archive staff is able to use preservation reports generated by OUDA to make
preservation decisions.
• The OU Archive is able to offer a digitise-on-demand service for video and audio
content to staff through OUDA.
• The OU Archive is able to offer a service to OU staff to preserve and make
accessible their digital content, if it meets selection criteria.
• OUDA is used to promote the OU Archive, Library Services and the OU itself to the
wider public.
The current assessment plan will not address these longer-term final objectives. They
belong to Phase Two implementation planning and assessment. We list them here
because Phase One assessment objectives must be understood in their context, in order
that they strive to pave to way to their conformance. That is, Phase One outcome
fulfilment will set the stage for justifying and enabling Phase Two continuation. OUDA’s
overall final success hinges on this transition.
For the current phase we must look at the intermediary goals that within two years our
small set of resources are actually capable of producing, such that we at least can
convince higher-level stakeholders that is continued support is justified and on its way to
delivering on the longer-term outcomes that its development is meant to serve.
As previously discussed, our chief goal for Phase One planning, marketing and assessing
this prototype is encompassed in the statement below:
The Open University Digital Archive (OUDA) has the following service goals:

59

• Capture OU history, learning, teaching and research materials to enable their
preservation, discovery and reuse, inform pedagogical decisions and facilitate
educational research.
• Complement existing platforms (such as OpenLearn, iTunesU and Study at the OU)
to support students’ informal to formal learning. Where assets already appear online
OUDA will drive traffic to the relevant platform for access through the use of
semantic technology, whilst aiming to preserve a high quality copy and associated
metadata in OUDL for the long-term.
• Contain a selection of fully-searchable digitised and ‘born digital’ OU materials from
videos and images to digitised documents; thus providing access to archive
materials previously only accessible by visiting the OU Archive, and supporting the
enquirer and study experience.
• Manage and preserve digital content in a long-term, sustainable manner in
accordance with OU policy and international preservation and metadata standards.
The following reproduces the six outcomes (seen above) by which we will measure the
success of OUDA after this initial phase one of planning:
1. A preservations policy plan is in place and has been approved by the OUDA
Steering Group. This preservation policy plan will contain all of the elements
described in the literature reviews above, and will be benchmarked against other
successful institutional policy directives.
2. OUDA will comply with the components of the TRAC audit that fall within the scope
of Phase 1 planning. That is, TRAC Sections A (Organizational Infrastructure) and
C (Technologies, Technical Infrastructure, and Security). Section B contains
elements of cataloguing ingest workflows, and will belong to Phase 2.
3. OUDA will contain prototypes of various content types. This will include complete
records and content of course material (i.e., video, audio, texts, supplementary
materials, books) for at least 3 non-current online legacy courses. This material will
cohere as part of a course, but also as materials that cohere as collections in
themselves (e.g., a collection of BBC videos, each video which is also “part of” one
or more course modules, or not).
4. The OU Archive study material records (metadata) are discoverable via OUDA.
They allow for effective searching, browsing and faceted filtering to discover content
and content types. That is, the content is usable to a user insofar as it is relevant, in
a readable format, reliably discoverable, and is at the correct level and coverage
specified.
5. OUDA can be used to guide people towards content related to their search on other
platforms such as iTunes-U and OpenLearn using linked data functionality. That is,
it will have linked data functionality through a SPARQL endpoint and interact with
the URIs minted for course materials at data.open.ac.uk.
6. OUDA has incorporated feedback from its most relevant external stakeholder
groups. That is, the OUDA prototype has received positive feedback from

60

stakeholders across groups regarding its potential ROI for increasing course
material re-use and course production productivity. And its potential value and utility
for enhancing OU’s academic and research reputation. In other words, OUDA
content is useful.
Assessment plan for outcomes (with relevant literature)
The following assessment rubric is a practical reference summary of our six (6) outcomes
listed above. It includes a general outline of the developmental sequence of each
outcome. That is, it describes in general terms performance indices as objectives for which
each Phase One outcome can be considered as having been initiated (beginning), as on
track for being properly developed (developing), and as having achieved a sustainable
(sustaining) level upon which Phase Two implementation can build upon.
Assessment rubric
Beginning Developing Sustaining Data Source
1.
Policy plan
in place
Research into policy
plans for possible
benchmarking
comparison policies
Have a draft produced and
distributed that contains
essential elements (from
literature reviews)
Policy is approved
across the institution and
guides digital repository
service practice.
Input from
relevant
stakeholders;
benchmarking
reviews
2.
Trusted
Repository
Audit
(TRAC)
TRAC audit checklist
lays foundation for
repository development
Section A Organizational
Infrastructure) and Section
C (Technical Infrastructure
and Security) are
documented with evidence
Section B (Digital Object
Management) is
complete with evidence.
TRAC external audit has
been initiated.
TRAC Audit with
evidence
3.
Digital
object
prototypes
established
Course material being
digitized, metadata
structure profiles
established, and
repository infrastructure
in place for step-wise
prototyping
One complete course has
been imperfectly
implemented within the
Fedora infrastructure
(including its audio-visual
and textual components).
Three complete courses
have been implemented.
Fedora
infrastructure
contains
harvestable
metadata and
data packages
4.
Repository
interface
established
Have begun web CMS
interface
implementation to
make API calls to the
Fedora data packages
Data objects and metadata
for prototype course are
organized according to
course. Search index and
faceted browsing
implemented.
Digital repository objects
are discoverable,
browsing and search
function, results appear
as desired, and content
becomes useful
Usefulness and
usability studies
5.
Metadata is
linkable
Metadata profiles
contain relational data
attributes (e.g., linked
data); Fedora RELS-
EXT datastream
functioning
Fedora FOXML
datastreams have RELS-
EXT data established,
linking resource attributes
(e.g., video isPartOf course
module)
RELS-EXT data follows
KMi data.open.ac.uk
data standards, and can
be linked to or from.
SPARQL endpoint
established.
Can write
SPARQL queries
against data to
pull it in from
external sources
6.
Stakeholder
feedback on
usefulness
Few stakeholders
(<25%) believe OUDA
will increase digital re-
use in course module
creation (or increase
ROI)
More stakeholders (>50%)
believe OUDA will increase
digital re-use in course
module creation (or
increase ROI) average
Most
stakeholders (>80%)
believe OUDA will
increase digital re-use in
course module creation
(or increase ROI)
average
Surveys, iterative
prototyping
demonstrations,
web analytics.
Will vary with
stakeholder type.

61

Additionally, the rubric indicates in general terms the source of data (data source) against
which these performance outcomes will be assessed. In what follows, we will review each
of these outcomes (1-6) in more detail below. In so doing, we will weave some of the
literature (from the literature review documentation above) into this detail, as at times the
literature provides effective recommendations regarding our selection of assessment tools,
methods, decision-outcome indicators, ways of analysing and reporting the data as
evidence.
Outcome 1: Policy plan in place
Preservation cannot merely be viewed as an issue of enabling and configuring the correct
software implementation. Wilczek and Glick (2006) write:
It seems obvious that no existing software application could serve on its own as a
trustworthy preservation system. Preservation is the act of physically and
intellectually protecting and technically stabilizing the transmission of the content
and context of electronic records across space and time, in order to produce copies
of those records that people can reasonably judge to be authentic. To accomplish
this, the preservation system requires natural and juridical people, institutions,
applications, infrastructure, and procedures.
However, in their “Survey of Institutional Readiness,” Kenney and Buckley (2005) found
only about half of 114 organizations with digital repositories had preservation policies
written, while a mere third had vetted and implemented these across management levels.
The same survey indicated that only 38% of these organizations reported they had
sustainable funding or some other form of ongoing commitment to the long-term
maintenance of the repository.
The first outcome (1) against which OUDA will be assessed in whether it has a repository
governance policy plan in place. Sheldon (2013) provides access to a taxonomy of 33
various digital preservation policies across museums and university library digital
collections. These will be used to benchmark our policy plan. A JISC commissioned study
(Beagrie, Semple, Williams et al., 2008) distinguishes between high-level preservation
policies and a preservation action plan.
High-level policy indicators will include elements of organizational viability, integration and
support of preservation objectives as part of organizational strategy, collection
development delimitations, procedural and financial accountability and responsibility.
Preservation action plan indicators (summarized from Kenny & Buckley, 2005; and
Beagrie, Semple, Williams et al., 2008) include:

62

Outcome 1 Repository governance policy plan in place
Target Audience PVC-LTQ and OUDA Steering Committee (led by NW, Library Director) must be
satisfied to achieve continued Phase Two funding; this will be evidenced by the
following:
Methods and Tools
for Evidence
High Level Policy and Action Plan indicators (see above) are assessed against
industry wide benchmarks (see Sheldon, 2013) for preservation planning. Policies
will be written by university archivist (RC), assessed/reviewed by OUDA Managers
(RN and LM), and signed off by Library Director. Also, the policy itself will be sent
to various archivist colleagues for informal review based on the following points
below:
Analysis of Evidence
(Data Plan)
Meeting preservation policy indicators occur alongside repository infrastructure
development, as the policy will guide and insure long-term infrastructure
sustenance and viability, the infrastructure being critical to meeting the goals of the
long-term preservation of the digital objects it holds.
Policy will be able to guide collection development criteria for inclusion & exclusion,
and eventually (in Phase Two) set the stage for submission, cataloguing, object
format guidelines, authentication mechanisms, procedures for content quality and
information package control, and object persistence and integrity information. It will
set the stage for sustaining this development in order that in Phase Two the TRAC
Section B (Digital Object Management) audit can be started.
Policy is able to provide a sustenance plan for its financial viability over the long-
term and, given the infrastructure, library staff are trained and able to maintain it.
Alternative Methods
and Tools
TRAC Section A: Organizational Infrastructure (governance, staffing, policy,
licensing, and financial sustainability). See Outcome 2.
Decision criteria Evidence will include a checklist of policies and their implementation level within
repository infrastructure. A satisfactory policy will have satisfied the university
archivist according to the “analysis of evidence” criteria above, as well as other
informally addressed archivist colleagues.
Recommendations
for Reporting
The report will follow preservation policy benchmarking examples from other
university digital archives.
Responsible Parties OU Archivist, Metadata Manager, OUDA Managers, and Associate Director (GN)
Timeline High-level policy (strategy etc.) development should occur during Phase 1a (project
definition and scope) and 1b (project planning). Draft versioning of this policy will be
submitted every 6 months (4 drafts) over the lifespan of Phase One (1a-1e).

63

Outcome 2: Trusted Repository Audit (TRAC)
The second outcome (2) has a twofold purpose. It employs the TRAC (Trustworthy
Repositories Audit and Certification28
) audit, a checklist of the necessary evidence
required to demonstrate long-term repository viability. It is composed of three sections, two
(A and C) of which we will use as assessment criteria:
sustainability)
management)
The organizational infrastructure (A) of OUDA will serve as an alternative method used to
expand upon and formally audit the policy reports generated through Outcome 1 (see
above). However, the TRAC audit includes wider application framework for assessing how
OUDA embodies both the organization’s policies and procedures, including but not
restricted to the following kinds of organizational infrastructure indicators29
:
• Governance
• Organizational structure
• Mandate or purpose
• Scope
• Roles and responsibilities
• Policy framework (see Outcome 1)
• Funding system
• Financial issues, including assets
• Contract, licenses, and liabilities
• Transparency
The technical infrastructure and security (C) of OUDA will serve as the main method of
assessing OUDA’s technical capacity to deliver on its promise to ensure long-term
preservation and easy access to the Open University’s digital objects making up its legacy
course modules. The criteria in Section C measure the adequacy of the repository’s
technical infrastructure, based on best practices for data management and security, to
satisfy digital object management and security demands of the repository and the digital
objects it stores. The kinds of technical infrastructure and security indicators and
requirements are roughly grouped as follows30
:
• General software system infrastructure requirements, including backup functionality,
hardware/software change control processes, and access and auditing control
metadata
• Digital object location, checksum identity and integrity control, backup,
synchronization, bit corruption, failsafe procedures, object testing, and ingest and
rebuilding capacities
• Data and software security and disaster and recovery preparedness planning

28
29
http://www.crl.edu/sites/default/files/attachments/pages/trac_0.pdf

30

64

Outcome 2 Trusted repository audit (TRAC), Section A and C
Target Audience The OUDA Steering Committee (led by NW, Library Director), and the entire library
staff (especially the cataloguers and archive staff).
The wider HE and library communities, and the digital preservation and curation
communities. TRAC is an internationally accepted audit, developed by the
OCLC/RLG Programs and the National Archives and Records Administration.
Currently the repository checklist is under the remit of the Center for Research
Libraries (CRL) group.
Methods and Tools
for Evidence
TRAC is a checklist of statements each of which must be addressed with tangible
forms of evidence in order to pass or fail each statement. The TRAC audit
documentation explains each point on the checklist, and provides illustrative (not
prescriptive) examples of evidence that show how the repository is poised to meet
each respective criterion. An example is reproduced below, as part of Section A:
A2.1 Repository has identified and established the duties that it needs to
perform and has appointed staff with adequate skills and experience to fulfill
these duties.
The repository must identify the competencies and skill sets required to operate the
repository over time and demonstrate that the staff and consultants have the range
of requisite skills—e.g., archival training, technical skills, and legal expertise.
Evidence: A staffing plan; competency definitions; job description; development
plans; plus evidence that the repository review and maintains these documents as
requirements evolve
(Data Plan)
TRAC includes a section, adapted from the Digital Curation Centre (DCC)
31
, on
articulating the applicability of each criteria in terms of 1) documentation as
evidence, 2) transparency (repository with its design specifications, policies and
procedures exposed for assessment), 3) adequacy (doing that which it is supposed
to do), and 4) measurability (as objective as possible with transparent indicators of
trustworthiness based on the documentation of adequacy).
Alternative Methods
and Tools
ISO 16363 (Trusted Digital Repository, TDR, Checklist) is as of 2012 an alternative
framework that is based on and grew out of TRAC. It is roughly similar to TRAC but
is recently gaining acceptance in the preservation community as an emerging
replacement for TRAC.
Decision criteria See “Analysis of Evidence” (Data Plan). TRAC is first applied as a self-audit. Later,
after Phase Two is complete, TRAC also includes the option of allowing external
auditors to evaluate the evidence provided using metrics based on 1 to 5, to gain a
formal TRAC certificate.
Recommendations
for Reporting
TRAC includes a formalized checklist and suggested kinds of documentation as
evidence required to satisfy each metric.
Responsible Parties OU Archivist, Metadata Manager, Systems Developer, Project Manager, OUDA
Managers (RN & LM), Associate Director (GN) and Library Director (NW)
Timeline Rolling: TRAC audit items should be included within the general project
management work breakdown structure timeline.
However, preservation actions will be documented across Phase 1c to 1e work
packages. Most work here will concentrate on TRAC Section A (organizational
policy), many points of which will be covered in Outcome 1 (above).
As infrastructure is developed, TRAC Section C will follow infrastructure

31
http://www.dcc.ac.uk/

65

development actions. Towards later phases, as the content and collections are
exposed and developed, TRAC Section B must be considered. This will occur
across Phase 1a to Phase 1e.
Outcome 3: Digital object prototypes established
The success of OUDA in Phase One implementation hinges on two important aspects:
firstly, preservation policy and repository infrastructure (Outcomes 1 and 2); and secondly,
digital object access and content re-use (Outcomes 4, 5, and 6).
Outcome 3 fits some where between these two. It describes a set of indicators that assess
the development of the collection itself and the digital content making it up; that is, the
digitization and organization of the digital content itself, and the metadata descriptions
based on the profiles and schema that structure it as linkable and universally accessible
objects.
A well-known evaluation framework, the interaction triptych framework (Fuhr et al., 2007),
outlines three main dimensions upon which digital library repositories can be assessed.
The model is reproduced below:
Through an analysis of factors (see also the DELOS32
assessment framework, which Fuhr
leads), he summarizes three dimensions important for evaluation: 1) performance, 2)
usefulness, and 3) usability. For our purposes here, Outcome 3 describes the interaction
of the repository system itself (see Outcome 2) with the content it stores, or what Fuhr
calls the performance factor. In general, by this we want to assess how well the software
repository system interacts with and is able to organize the formats, structures and
representations of the digital content it is intended to preserve (i.e., the metadata and the
digital objects the metadata describes).

32
http://www.delos.info/index.php?option=com_content&task=view&id=26&Itemid=52

66

Fuhr also describes other parameters of performance evaluation that, while certainly
relevant to system performance, will fall outside our scope of evaluating outcomes. They
mainly deal with response-times, precision and recall factors taken from studies in
information retrieval. These are important, but will fall within the scope of Phase Two.
The target audience for OUDA in Phase One will be content producers, mostly highly
skilled digital users. For this phase, content producers will provide feedback on the
conceptual models and later the wireframes and prototypes developed that represent the
way in which the system will structure the digital content, as well as the formats in which
this digital content is stored. This assessment will be iterative and relatively informal for
Phase One, as described in the marketing documentation.
That is, conceptual and working prototypes of the service will 1) communicate and
promote (i.e., market) the kinds of digital objects and the ways in which they will be
structurally organized, and 2) engage these users to assess these proposals, or re-assess
them after previous feedback has been integrated.
Outcome 3 Digital object prototypes established
Target Audience All library stakeholders, but mostly the university archive teams. PVC-LTQ for
continued funding. External stakeholders interested in content re-use and course
module inspiration (OMU, LTS and OU Academic Units) to achieve greater
productivity and increase ROI.
Methods and Tools
for Evidence
The assessment will focus on the iterative feedback loops during marketing
communication with various stakeholder groups (i.e., content and module
production teams). These stakeholders will be presented with conceptual plans,
wireframes, and later working prototypes that illustrate the kinds and structure of
content, working models of its organization and presentation, and the strategic
ways OUDA will assist their respective workflows (or strategic value of the
university, in the case of high-level managers). Assessment will involve the
informal feedback from these relatively informal communication actions.
The assessment must also occur internally to ensure the repository is structured
based on metadata and repository standards and best practices. These will occur
as part and parcel of weekly project meetings. The assessment plan will involve the
continual testing and quality control of the metadata and whether or not it structures
the digitized course modules within the Fedora repository; and whether the web
application is able to serve up this structure in ways that reflect its desired
organization according to this feedback.
(Data Plan)
Evidence will consist of a viable “proof of concept”. It should include the availability
of three (3) complete legacy course modules, and the digitized content it delivered.
For instance, it will include “Science 100” (OU’s first module delivered in 1972), and
the audio-video-textual objects this course delivered to students. This will include
descriptive metadata profiles for all object types (EBUCore, MODS, XCRI, VRA,
and the FOXML container class).
The project team will assess the structural integrity of data packages within the
Fedora digital preservation software system. While the digitization of materials and
metadata profiling are necessary components, at stake is how they are packaged
as XML data objects within Fedora, and their implementation within the web
application servers in order that front-end API arguments can compile these objects
upon client-requests. For instance, a video must exist as an object in itself, but
also co-exist as part of various course modules (i.e., for re-use), which itself is
described by course module metadata (XCRI).

67

Alternative Methods
and Tools
NA
Decision criteria Decisions here will be based on our feedback loop with stakeholders as discussed
in the marketing documentation (i.e., academics and course module producers
provide feedback regarding the best ways of organizing this content in ways that
improves and inspires their course production work flows/habits). As such, each
prototype of the service, as developed in iterative fashion, should reflect this
feedback. All decisions must abide by industry-wide standards and best practice.
Recommendations
for Reporting
Internal reports only. Assessment outcomes incorporated into weekly project
meetings and infrastructure requirements. Blog reports to update interested parties
(for transparency).
Responsible Parties OU Archivist, Metadata Manager, Systems Developer
Timeline Iterative. Most work will occur during Phase 1c during WP1 (infrastructure) and
WP2 (content) work packages are being completed. Work will continue into Phase
1d during which the content preservation model must adhere to both Outcome 2
(TRAC preservation policy) and expose the content model according to Outcomes
4 and 5 (interface usability and content usefulness).
Outcome 4: Repository interface established (and is usable)
There is a lot of research devoted to the study of digital library evaluation criteria.
Chowdhury and Chowdhury’s (2003) book on digital libraries is a good place to start. One
constant consideration for a university library is to provide for users with special needs
(Kwak & Bae, 2005). Related to this is research by Inskip, Butterworth, and MacFarlane
(2008) that has found that usefulness and usability is highly dependent on the user’s level
of research experience. Thus, while OUDA target audience in the initial phase will be
content producers—or faculty and staff within the university, presumably with higher levels
of research experience, our long-term goals include students and to some extent the
general public. One study by Kengeri, Seals, Harley, Reddy, & Fox (1999) found very few
differences between novice and experienced users of digital libraries, and Theng et al.
(2008) even noted that children designed systems incorporating the same usability
features that adults typically desire.
We must ensure the interface is usable. The triptych model for Outcome 3 assesses how
the content interacts with the system. Outcome 4 will assess how the user interacts with
the system, or its ease-of-use, use of navigation and terminology (i.e., data labels), and
learnability. We will employ methods such as card-sorting and focus groups to determine
navigation flow, content-to-page organization, and preferred design elements. A study by
Tsakonas and Papatheoudou (2006) found that users prefer a system that is useful over
usable, insofar as its primary goal is to locate content within a system. However, research
by Xie (2006) found usability ranked higher. We choose to prioritize both, and believe
usability will influence usefulness more than vice-versa. Outcome 5 and 6 will assess
linked data functionality and usability.

68

Outcome 4 Repository interface established (and is usable)
Target Audience All library stakeholders. PVC-LTQ for continued funding. External stakeholders
interested in content re-use and course module inspiration (OMU, LTS and OU
Academic Units) to achieve greater productivity and increase ROI.
Methods and Tools
for Evidence
Methods and tools will consist of card-sorting and focus groups
(Data Plan)
The OUDA team has definite ideas regarding how the interface will look and feel.
Indeed, in many respects, we are modeling OUDA after the recently released
London School of Economics Digital Library
33
. Our design will also follow our
planned implementation of how we understand the content needs of content
producers and OU academics. Our card-sorting and focus group tools will focus
more on the details of aesthetics (colors, fonts, contrasts, borders, white-space),
user-flow (e.g., the use of breadcrumbs, menu items, back-tracking), ease-of-use
(learnability, desire-to-continue, help text, browse vs. search functionalities), and
general page orientation (block designs and clutter).
The look and feel of this will form the basis again of a “proof of concept,” essential
for continued funding, as PVC-LTQ and Steering Group most quickly swayed by
what they can immediately see, not the underlying data structure and integrity of
the digital objects and metadata making it up.
Alternative Methods
and Tools
Eye-tracking software is available, but beyond our current budget. Also, controlled
studies might be considered, but these take alternative designs to use as
comparison groups, and our back-end systems developer is also our front-end
systems developer.
Decision criteria Decisions here will be based on the evidence collected in the card-sorting and
focus group sessions administered by the Content/Access Manager (KB). Minor
adjustments to the page will be evidence based, but remain within the constraints
of the site’s general structural purpose (to organize course modules and their
component digital multi-media objects). Changes to the interface will occur
iteratively
Recommendations
for Reporting
Internal reports only that summarize the results of the focus group and card sorting
methods. From this data, KB will iteratively develop wireframes to be informally
assessed by project team. Decisions will follow with tweaks to wireframes based on
evidence and team preferences. Resulting wireframes and other assessment
indicator data to be incorporated into weekly project meetings between the
Content/Access Manger and Systems Developer, who will implement wireframe
into ongoing web development workflow.
Responsible Parties Content and Access Manager (KB) & Systems Developer (JA). Blog reports.
Timeline Iterative. Will occur during second half of project, during Phase 1d work package,
and most intensely during Phase 1e (assessment and project closure)

33
http://digital.library.lse.ac.uk/

69

Outcome 5: Metadata is linkable
The results of this outcome do not require formal or informal assessment. The outcome is
based objectively on how the metadata is structured within Fedora. It is either structured
correctly, or it is not. Fedora reserves particular datastream types to RDF data that is
linkable to data.open.ac.uk, the OU’s internal SPARQL endpoint developed and
maintained by the OU Knowledge Media Institute (KMi). Success on this outcome is
dependent on whether the metadata profiles established contain relational data attributes
that link to data.open.ac.uk. The details are technical. The main individuals responsible for
this outcome are the systems developer (JA) and the metadata developer (LW).
One objective source of data that will be used to evaluate outcome success is the ability to
write SPARQL queries against this set of linked data in order to pull it in as “related
material” from external sources. For instance, if the OUDA presents a course module on
the work of William Shakespeare, and a video lecture is accessed that belongs to this
module, linked data will allow for OUDA to automatically pull in similar content from either
the BBC, from Wikipedia (based on DBpedia), or even from within the OU itself, insofar as
the Open Media Unit stores other kinds of Shakespeare content that is also linked to
data.open.ac.uk.
Additionally, what this means is that any external organization (e.g., the BBC or any
university) that wishes to write SPARQL queries against the data stored within OUDA will
also be able to pull in OUDA digital objects as “related to” the content presented on their
screens.
Outcome 6: Stakeholder feedback on usefulness (STELLAR re-visited)
Outcome 3 assesses how the content and system interact, Outcome 4 assesses how the
user interacts with this system, and Outcome 6 assesses how the user interacts with this
content, or how useful the content is. Fuhr (2007) describes usefulness as a reflection of
“how users perceive the relevance of a DL [digital library] with their needs, the width, the
breadth, the quality, as well as the validity of its collection, and the ability to serve their
goals.”
Based on our analysis of user need through informal and iterative feedback gained during
our marketing meetings, we must determine how OUDA can deliver reliable and relevant
content in a format and at a level appropriate to best adapt into the workflow of OU course
module content producers. As each prototype is developed, Outcome 6 will assess the
potential usefulness of OUDA to course production.
However, the assessment during Phase One will remain largely theoretical, in ways not
entirely dissimilar to the results of the STELLAR project (non-published, 2013). When
stakeholders across the university spectrum (academic, staff, senior administrators, asset
managers, content producers) were surveyed, their responses showed high levels of
stakeholder interest and recognised need for the provisioning of preservation services for
non-current learning material. This need included mention that easy access to non-current
learning material would inspire academics and content producers when producing new
content; and also that it would serve university strategic value by eliminating unnecessary
duplication by encouraging course material re-use.

70

That is, the results gave high indication of potential usefulness of concept. During Phase
1e at the end of Phase One, we plan to re-use the STELLAR survey in modified form. That
is, stakeholders will be able to log-on to the OUDA interface and search and browse the
course module content itself. That is, they will use the site to search and browse the three
course modules and the audio-visual and textual digitized content each of the modules
contains. Obviously, browsing a mere three course modules is not very useful it itself, but
respondents will be asked to envision Phase One implementation as a soft-launch
prototype of how OUDA will provision the collection of all course modules, and assess its
potential usefulness as such. Questions will be framed in much the same way as the
STELLAR profile, except will be reframed to account for this soft usage.
Outcome 6 Stakeholder feedback on usefulness (STELLAR re-visited)
Target Audience All sample of all stakeholder groups, including a small set of senior-level university
managers, including the PVC-LTQ.
Methods and Tools
for Evidence
This work will continue where the work of the STELLAR project left off (see
planning documentation for review). Method will consist of one formalized online
survey. The survey will address in modified form nearly all survey questions
addressed in the STELLAR study in order to provide pre-post test comparison
results. The questions will be modified to reflect the fact that each user has actually
used the OUDA online preservation web portal, and searched and browsed the
three (3) course modules it contains (yet in no pre-determined manner, but instead
as fits their fancy), but for a specified amount of time (e.g., they will be asked to
navigate around OUDA for a period of 10 non-interrupted minutes; or, they may be
asked to find certain kinds of content).
Additionally, we will include another set of questions that address the usability and
usefulness of the web experience itself. That is, questions that have a bearing on
the way OUDA is currently designed to expose OU content per se. It will include
open-ended questions that address aspects in which the stakeholder feels OUDA
could be improved, requires change, is unpleasant to use, and necessary issues
OUDA fails to address given OU wide course production workflow realities.
(Data Plan)
The survey will administer a set of Likert-scale questions that have various
responses that span from 1 (agree) to 5 (disagree). This will reflect the same
structure of STELLAR results, and the two paired question sets (pre- and post-
OUDA soft launch) can be compared and contrasted, with the expectation that
interest in the content OUDA provisions would remain equally as high given the
proof of concept. If necessary, Chi-squared tests can evaluate pre vs post
response variation.
The open-ended questions will be content analyzed by a set of coders in attempt to
classify these into an abstract set of common categories.
Alternative Methods
and Tools
Follow up in-person focus sessions might penetrate further into particular
responses provided by given respondents. While the questionnaire will remain
anonymous, respondents will be given the opportunity to reveal themselves by
signing a waiver at the end of the survey, from which we can unlock the key
identifying the respondent to the very survey he/she filled out. In this way, if we find
particular responses enlightening and wish to hear more, we can contact that
individual and request further meetings to discuss issues more concretely.
Decision criteria Given that responses from STELLAR were phenomenally high, regression to the
mean usually implicates the fact that we would expect user groups to be less
enthusiastic about OUDA as a service. This is expected. This will be exacerbated
by the fact that OUDA Phase One will remain an incomplete product in regards to

71

its final service goals and outcomes (after Phase Two). For that reason, we deem
responses that are 20% less than STELLAR outcomes as positive. That is, unless
said outcome is less than 60% approval in terms of absolute value. Items that
decrease to 40-50% approval will be earmarked for critical review within the OUDA
Steering Group. If more than 50% of the values achieve less than 50% approval
value, the continued sustenance of OUDA will have to be seriously questioned. In
this case, follow up focus groups with members of external stakeholders to identify
the main issues will be critical.
Recommendations
for Reporting
Assessment outcomes will be internally published in a formal report (50+ pages)
describing in detail the results of the survey. Summaries of the findings will also be
published on the OUDA blog and other internal communications.
Responsible Parties OUDA Content (LM) and Services Manager (RN) will develop and administer the
online survey. The OUDA Steering Group and the Library Director must approve
the survey prior to administration.
Timeline Survey will be administered during the last month of Phase 1e. Its results will be
analyzed and reported on in the month that follows.
Reflection
Phase One focuses on developing the software repository system to appropriately
structure OU’s digitized course materials according to international metadata standards
and in ways that ensure the content and data are preserved over the long-term according
to a certifiably trustworthy standard. Long-term preservation is at heart the main goal of
OUDA. While preservation itself as a practice might be questioned, a preservation system
that simultaneously provides useful and usable content increases value to course module
producers by decreasing module production time (e.g., through content re-use).
Without the infrastructure in place, however, it is impossible to directly assess whether this
strategic reasoning is supported by actual content re-use, insofar as non-current OU
learning material can be recycled, insofar as past course accessibility serves to inspire
future course creation, and insofar as the entire collection of past courses itself serves to
contextualize the history and enhance the reputation of OU’s model of higher online
education.
These direct outcomes will provide a clearer framework upon which to assess OUDA
during Phase Two. Assessing a service usually focuses on service users as the primary
source of evidence to measure its impact. However, for Phase One, we had to use many
measures derived not from studies of direct user assessment. Instead many of the
outcomes addressed in this document assessed policy and infrastructure conformance to
a set of externally defined standards and benchmarks. That is, measuring whether a policy
conforms (or not) to a set of benchmarks does not involve the assessment of users. The
same is true for assessing whether the infrastructure of OUDA conforms to TRAC auditing
standards. Nevertheless, without a set of preservation policies, and without a preservation-
trustworthy infrastructure, later studies that can directly assess the usefulness of OUDA as
an accessible system for course module production would be rendered impossible.

72

For this reason, in the case of OUDA, as I believe may be the case for many software
projects, technical infrastructure development was difficult to adapt to the basic framework
of service-user assessment as defined in IST 613. For Outcomes 4 and 6 in OUDA, the
framework works fine. For this reason, I think of Outcomes 1, 2 and 3 as more akin to
interim outcomes; that is, as outcomes that will eventually, in a later phase, indirectly
impact service-users in ways that enable a different (but related) set of outcomes. This is
already true insofar as Outcome 1 and 2 enabled us to assess actual users regarding
Outcome 6.

Bibliography
Audit, T. R. (2007). Certification (TRAC): Criteria and Checklist. Center for Research
Libraries, OCLC Online Computer Library Center. Retrieved February 2, 2014, from
Akeroyd, J. (2005). Information management and e-learning: some perspectives. Aslib
Proceedings: New Information Perspectives, 57(2), pp. 157-67. doi:

10.1108/00012530510589128

Ball, A. (2010). Preservation and Curation in Institutional Repositories. Version
1.3. University of Bath, UK: Digital Curation Centre. Retrieved February 2, 2014, from
http://www.dcc.ac.uk/sites/default/files/documents/reports/irpc-report-v1.3.pdf
Baily, C. W. (2011). Institutional Repository and ETD Bibliography 2011. Houston: Digital
Scholarship, 2011. Retrieved February 2, 2014, from http://digital-
scholarship.org/iretd/iretd2011.htm
Baily, C. W. (2011). Digital Curation and Preservation Bibliography 2010. Houston: Digital
Scholarship, 2011. Retrieved February 2, 2014, from http://digital-
scholarship.com/dcpb/dcpb2010.htm
Beagrie, N., Semple, N., Williams, P., & Wright, R. (2008). Digital preservation policies
study. JISC. Retrieved February 2, 2010
from
http://www.jisc.ac.uk/media/documents/programmes/preservation/jiscpolicy_p1finalreport.p
df

Beagrie, N., Bellinger, M., Dale, R., Doerr, M., Hedstrom, M., Jones, M., ... & Woodyard,
D. (2002). Trusted Digital Repositories: Attributes and Responsibilities. Research Libraries
Group & Online Computer Library Center, Report. Retrieved February 2, 2014, from
http://oclc.org/research/activities/trustedrep.html
Becker, C., Kulovits, H., Guttenbrunner, M., Strodl, S., Rauber, A., & Hofman, H. (2009).
Systematic planning for digital preservation: evaluating potential strategies and building
preservation plans. International Journal of Digital Librarianship, 10(4), 133-157. doi:
10.1007/s00799-009-0057-1
Chowdhury, G. G., & Chowdhury, S. (2003). Introduction to digital libraries. London: Facet
Publisher.
Collier, M. (2010). Business Planning for Digital Libraries: International Approaches.
Leuven University Press.
Consultative Committee for Space Data Systems. (2012). Reference model for an open
archival information system (OAIS). Washington, D.C.: CCSDS Secretariat. Retrieved
February 2, 2014 from http://public.ccsds.org/publications/archive/650x0m2.pdf
Cullen, R. J., & Calvert, P. J. (1995). Stakeholder perceptions of university library
effectiveness. The Journal of Academic Librarianship, 21(6), 438-448.

74

Davis, P. M., & Connolly, M. J. (2007). Institutional repositories: evaluating the reasons for
non-use of Cornell University's installation of DSpace. D-lib Magazine, 13(3/4). Retrieved
February 2, 2014 from http://www.dlib.org/dlib/march07/davis/03davis.html
De Rosa, C., Cantrell, J., Cellentani, D., Hawk, J., Jenkins, L., & Wilson, A. (2005).
Perceptions of Libraries and Information Resources: A Report to the OCLC Membership:
Dublin, OH: OCLC. Retrieved February 2, 2014 from
http://oclc.org/reports/2005perceptions.en.html
Fay, E. (2010). Repository Software Comparison: Building Digital Library Infrastructure at
LSE. Ariadne, 64. Retrieved February 2, 2014 from http://www.ariadne.ac.uk/issue64/fay/
Ferreira, M., Rodrigues, E., Baptista, A. A., & Saraiva, R. (2008). Carrots and sticks: Some
ideas on how to create a successful institutional repository. D-Lib Magazine, 14(1), 3.
Retrieved on February 2, 2014 from
http://www.dlib.org/dlib/january08/ferreira/01ferreira.html#8
Fuhr, N., Tsakonas, G., Aalberg, T., Agosti, M., Hansen, P., Kapidakis, S., ... & Sølvberg,
I. (2007). Evaluation of digital libraries. International Journal on Digital Libraries, 8(1), 21-
38. doi:10.1007/s00799-007-0011-z
Inskip, C., Butterworth, R., & MacFarlane, A. (2008). A study of the information needs of
the users of a folk music library and the implications for the design of a digital library
system. Information Processing & Management, 44(2), 647-662. doi:
10.1016/j.ipm.2007.05.004
Jantz, R, & Giarlo, M. J. (2006). Architecture and technology for trusted digital repositories.
D-Lib Magazine, 11 (6). Retrieved February 2, 2014 from
http://www.dlib.org/dlib/june05/jantz/06jantz.html
Jeng, J. (2013). What is usability in the context of the digital library and how can it be
measured. Information technology and libraries, 24(2), 47-56. doi: 10.6017/ital.v24i2.3365
Joint, N. (2010). The one-stop shop search engine: a transformational library technology?:
ANTAEUS. Library Review, 59(4), 240-248. doi: 10.1108/00242531011038550

Kaczmarek, J., Hswe, P., Eke, J., & Habing, T. G. (2006). Using the Audit Checklist for the
certification of a Trusted Digital Repository as a framework for evaluating repository
software applications. A progress report. D-lib Magazine,12(12), 1082-9873.
doi:10.1045/december2006-kaczmarek

Kenney, A. R., & Buckley, E. (2005). Developing digital preservation programs: The
Cornell survey of institutional readiness, 2003-2005. RLG DigiNews, 9(4). Retrieved
February 2, 2014 from
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070519/viewer/file1088.html#article0
Lavoie, B. F. (2004). The open archival information system reference model: Introductory
guide. Microform & imaging review, 33(2), 68-81. Doi: 10.1515/MFIR.2004.68

LeFurgy, W. G. (2002). Levels of service for digital repositories. D-Lib Magazine, 8(5).
Retrieved February 2, 2014 from
http://www.dlib.org/dlib/may02/lefurgy/05lefurgy.html

75

Marill, J. L., & Luczak, E. C. (2009). Evaluation of digital repository software at the national
library of medicine. D-Lib Magazine, 15(5/6), 1082-9873. Retrieved February 2, 2014 from
http://www.dlib.org/dlib/may09/marill/05marill.html
Kengeri, R., Seals, C. D., Harley, H. D., Reddy, H. P., & Fox, E. A. (1999). Usability study
of digital libraries: ACM, IEEE-CS, NCSTRL, NDLTD.International Journal on Digital
Libraries, 2(2-3), 157-169. doi: 10.1007/s007990050044
Kuhlthau, C. C. (1991). Inside the search process: Information seeking from the user's
perspective. JASIS, 42(5), 361-371. Retrieved on February 2, 2014 from
http://ptarpp2.uitm.edu.my/silibus/insidesearch2.pdf
Kwak, S. J., & Bae, K. J. (2009). Ubiquitous library usability test for the improvement of
information access for the blind. Electronic Library, The, 27(4), 623-639.
doi:
0.1108/02640470910979589
Mi, J., & Nesta, F. (2006). Marketing library services to the Net Generation.Library
Management, 27(6/7), 411-422. doi: 10.1108/01435120610702404
Prieto, A. G. (2009). From conceptual to perceptual reality: trust in digital
repositories. Library Review, 58(8), 593-606. doi:
10.1108/00242530910987082

Rousseau, G. K., Jamieson, B. A., Rogers, W. A., Mead, S. E., & Sit, R. A. (1998).
Assessing the usability of on-line library systems. Behaviour & Information
Technology, 17(5), 274-281. doi: 10.1080/014492998119346
Ross, S. (2012). Digital preservation, archival science and methodological foundations for
digital libraries. New Review of Information Networking, 17(1), 43-68. doi:
10.1080/13614576.2012.679446
Rubin, R. J. (2006). Demonstrating results: Using outcome measurement in your library.
American Library Association.
Suleman, H., & Fox, E. A. (2001). A framework for building open digital libraries. D-Lib
magazine, 7(12), 1082-9873. doi: 10.1045/december2001-suleman
Steinhart, G., Dietrich, D., & Green, A. (2009). Establishing trust in a chain of preservation:
The TRAC checklist applied to a data staging repository (DataStaR). D-Lib
Magazine, 15(9/10). Retrieved on February 2, 2014, from
http://www.dlib.org/dlib/september09/steinhart/09steinhart.html
Theng, Y. L., Nasir, N. M., Thimbleby, H., Buchanan, G., Jones, M., Bainbridge, D., &
Cassidy, N. (2000). Children as design partners and testers for a children’s digital library.
In Research and Advanced Technology for Digital Libraries (pp. 249-258). Springer Berlin
Heidelberg. doi: 10.1007/3-540-45268-0_23
Tsakonas, G., & Papatheodorou, C. (2008). Exploring usefulness and usability in the
evaluation of open access digital libraries. Information Processing & Management, 44(3),
1234-1250. doi: 10.1016/j.ipm.2007.07.008

76

Tsakonas, G., & Papatheodorou, C. (2006). Analysing and evaluating usefulness and
usability in electronic information services. Journal of information science, 32(5), 400-419.
doi: 10.1177/0165551506065934
Van House, N. A. (2002). Digital libraries and practices of trust: networked biodiversity
information. Social Epistemology, 16(1), 99-114. doi: 10.1080/02691720210132833
Xie, H. (2006). Evaluation of digital libraries: Criteria and problems from users'
perspectives. Library and Information Science Research, 28(3), 433-452. doi:
10.1016/j.lisr.2006.06.002
Glick, K., Wilczek, E., & Dockins, R. (2006). Fedora and the Preservation of University
Records Project*. RLG DigiNews, 10(5). Retrieved on February 2, 2014, from
http://dca.lib.tufts.edu/features/nhprc/reports/index.html.
Wirtz, J. (2012). Essentials of Services Marketing. FT Press.

Planning, Marketing and Assessing a Digital Library:The Open University Digital Archive (OUDA)

More Related Content

Similar to Planning, Marketing and Assessing a Digital Library:The Open University Digital Archive (OUDA) (20)

More from sstose (9)

Recently uploaded (20)

Planning, Marketing and Assessing a Digital Library:The Open University Digital Archive (OUDA)