By popular demand building a consortial demand driven program

Editor's Notes

• #2: Our panel is on a demand driven acquisitions pilot project that is a partnership among Orbis Cascade Alliance, EBL, and YBP.
• #3: I will provide an overview of our pilot project. Each of our panelists will describe an element of our project, along with lessons learned. We believe that our lessons learned are applicable to different consortia and individual libraries.
• #4: As mentioned this is a partnership between the three parties.
• #5: Orbis Cascade Alliance is a consortium of 36 academic libraries in Oregon and Washington. Institutions range in size from Chemekta Community College to University of Washington. The governance structure consists of a council of library directors that includes a representative from each institution. This body oversees all the Alliance’s activities, including policies, programs and the budget. The Alliance also has a Board of Directors. There are a variety of groups within the Alliance who manage and carry out different programs and initiatives set by the Alliance Council. The implementation team is a temporary team that is under the Collection Development and Management Committee (CDMC). A Collaborative Technical Services Team (CTST) is also participating in the demand-driven acquisitions pilot project through its ebook working group.
• #6: In late 2008 and early 2009, the Alliance Council developed a strategic agenda that led to this pilot project. Our pilot project contributes to two of the five elements of the strategic agenda, cooperative collection development and collaborative technical services. Along with several other proposals related to cooperative collection development, CDMC explored different possibilities of a shared ebook program. In spring of 2009, an ebook task force was charged to investigate the opportunities and challenges associated with a shared ebook program. The task force’s report and recommendations led to the creation of a new ebook team. Starting in early 2010, this new ebook team was charged to select an acquisitions model for ebooks. This team investigated different acquisitions models, issued an RFI to ebook aggregators and publishers, and selected a demand-driven acquisitions model with EBL as the vendor of choice. Last January at ALA Midwinter, members of the implementation team, including representatives from EBL and YBP, had its first meeting. This new team also included members of the CTST. Our implementation team initially set a go live date of May 2011. As a result of workflow issues, we delayed the start of the pilot project until July 2011. We have reached the midpoint of our pilot project. We expect to continue the project while funds exist, which we anticipate lasting until January or February of 2012. Next week, the Alliance Council will vote on a recommendation to extend funding through the remainder of this fiscal year. In January 2012, the implementation team will make a recommendation on whether this should turn into a permanent program and outline a more sustainable funding model.
• #7: The implementation team has met all its original goals for a shared ebook program. The participation of EBL and YBP has been critical to our success. We are also meeting our stated goal of participation by all 36 members. We are moving closer to the idea of one collection for 36 institutions through the ownership of ebooks by all the libraries. By selecting an aggregator, we provided content relevant to such a diverse group of libraries.
• #8: Another key component of our project was every library had to contribute funds to the pilot project. This model is based on our model for e-resources that are purchased through the Alliance. For a permanent program, we hope to use a combination of FTE, acquisitions budget, and usage from the pilot project.
• #9: .
• #14: .
• #15: As might be expected, the technical services environment across the Alliance libraries is complex. The consortia operates a shared WorldCat Local catalog, Summit. Some of the member libraries also utilize institutional instances of WorldCat Local, others maintain Innovative catalogs, some run both, and one library uses Evergreen. As such, the pilot’s implementation team knew that any successful workflow would need to speak to a diverse set of needs, resources, and expectations.
• #16: That said, the prospect of creating and managing 36 unique workflows was smartly deemed too time-consuming and expensive. Rather, the Implementation team aimed to deliver a single workflow that could be effectively utilized by all. To help identify the best process, the implementation team and the Alliance’s Collaborative Technical Services team did a lot of info gathering. They solicited libraryfeedback via survey. The team asked the vendors to detail the cataloging options, defining such details as the availability of OCLC control numbers, the cost, and the quality of records. Finally, being a WorldCat Local consortium, the Alliance consulted with OCLC.These efforts provided a map for the workflow’s requirements. The survey revealed, for instance, that the majority of libraries planned to promote discovery through Summit and local record loading and were prepared for OCLC batch record retrieval and loading. The Alliance’s Collaborative Technical Services Team led the communication with OCLC. Of primary concern was how discovery and access would be achieved via Summit, as there was not a locally hosted instance of the shared catalog. The team learned this could be addressed by using a new service, the WorldCat Knowledge Base, which allows libraries to set holdings for electronic collections, facilitating access via “one-click” links.
• #17: Taken together, the member library, OCLC, and vendor feedback advised the following requirements: The delivery of of OCLC WorldCat Cataloging Partners (WCP) records.  Use of OCLC Knowledge Base. Two Alliance holdings symbols, one to designate non-owned pilot content and one to designate purchased pilot content. A monthly record delivery schedule 
• #18: The resulting workflow included several defining features. It was very much focused on updating and tracking the status of the Knowledge Base. Twice monthly, EBL sends files to OCLC to facilitate the creation of access links in Summit. It was monthly, with the OCLC updating the KB and the end of each month followed by the delivery of marc records and it was batch based. OCLC made the marc file available via EDX, the Alliance’s Collaborative Technical Services team retrieved it and then distributed it to the member libraries via a central website. And, finally, those libraries loading records processed the files.  
• #19: As I’ve described, Knowledge Base was central to the activities and the schedule of the workflow. Largely, it successfully delivered the automated creation of “single-click” access links in Summit and in member library instances of WCL. For the most part, the Alliance, EBL, and YBP expected, planned for, and weathered the idiosyncrasies and surprises that come with any new product. Managing the workflow was not simple, however, for either the vendors or for the Alliance.It necessitated a large and time-consuming body of manual activities and workarounds. For example, for EBL, ensuring the monthly record orders only included titles with Knowledge Base links required a title-by-title review. For the Alliance, reviewing, trouble-shooting, and processing the monthly marc files involved the efforts of multiple team members and many hours of work. Looking toward the future and a post-pilot program, the workflow was unsustainable. Moreover, member library reporting revealed that the majority of intuitions were not relying on Summit or the KB to facilitate discovery and access to the pilot content. It this since, the focus of our efforts was off-base,Finally, while new content was added to the program on a weekly basis, discovery via Summit and the library catalogs was consistently one to two months delayed.In late September 2011, after an evaluation of the strengths and weaknesses of the workflow, the implementation team created a revisedprocess.Most significantly, the team chose to disconnect the delivery of records from the status of the Knowledge Base, and to receive weekly record deliveries. These changes were made to facilitate a timelier discovery service and to build the foundation for a sustainable, automated long-term workflow. The Knowledge Base remains an integral component of the process, but its status no longer dictates the timing of workflow’s activities. In this, the Alliance had to make a difficult compromise: without the careful timing of the original workflow, Summit access for some portions of the pilot content is broken – the needed links do not appear. This was accepted, however, as the new workflow better meets the discovery and access needs of most Alliance users.
• #20: While the original workflow proved to be problematic and was modified, the implementation team absorbed several valuable lessons. Initially, as I’ve said, the team underestimated the time and resources required. While this mistake was quickly discovered, the project would have benefited from an earlier understanding of this. The team also learned the importance of beginning with a sustainable workflow, rather than building a difficult to manage, manual processes, we hoped would be automated in the future. Activities should be prioritized to address majority requirements and expectations. While Summit is an important resource for the Alliance and the workflow still addresses its needs, it is no longer the central focus of the workflow, as most of the member libraries are not using it as a primary discovery interface. The changes the implementation team made to the workflow highlight the importance of flexibility. From the start, the team understood that modifications—in all areas of the project—might be needed in order to influence success. Finally and most importantly, the all of the teams activities and decisions were informed by a holistic understanding of the Alliance’s technical services environment. While it could have been used more effectively at times, without this data and its detail, the team would have been lost and the workflow irrelevant.
• #21: While there were no surprising lessons learned regarding communication, the experiences of the implementation team underline the importance of on-going management of information sharing. In the following section, we will take a quick look at how communications were handled through the Alliance structure; how training was planned and implemented and the marketing efforts pursued in support of the project.
• #22: The existing committee structure has provided the framework for reporting out at the committee, council and individual level.The implementation team reports back to the CDMC, CTST and the Board through existing channels. While this has worked well in general, adjustments have been made to address communication problems as they arose. As the project was moving from the planning to the implementation phase, it became clear that representation from each institution was needed to support information sharing between the group and member libraries. As a result, the DDA Liaison group was formed. This group is supported by a new listserv, through which the implementation team can request and disseminate information.One issue that wasn’t recognized until the project was underway was that the two groups working on the project, the implementation team and the ebook cataloging team, were working in parallel with not entirely clear charges. In hindsight, both teams could have worked more effectively had they met early on in the process to compare charges and discuss decision making. The workflow issues that surfaced during implementation required technically difficult conversations about records and this, at times, left the team overwhelmed. This combined with the lack of clarity of the teams’ charges lead to delays in making decisions and moving the project forward. When the implementation team began grappling with these workflow issues the decision was made to add the head of the Ebook cataloging teamto the implementation team. This addition to the team allowed more direct communications about the technical challenges both groups were confronted with. These issues bring into focus the need for team members who are willing to work with complex issues and stay focused on the task at hand. The implementation team greatly benefitted from one member, in particular, with the ability to take complicated information and make it comprehensible to the rest of the team. Finally, in addition to implementation team members and Alliance staff, both EBL and YBP were represented at every meeting. By including vendors in the process from the beginning of the project, the Alliance benefitted by having the vendor’s close attention and understanding of the project.
• #23: For the pilot project, the team decided against the use of end-user promotion. Issues considered in making this decisions included limiting variables to be evaluated, the size of the pool, and the nature of pilot projects. This decision raised questions among the public services community but appears to have been adhered to. Along with the other adjustments agreed upon in September, the ban on end-user marketing is now also being reviewed in light of the low usage statistics. It is possible that this policy will change for the final phase of the pilot. While the implementation team has worked hard to manage communication within the Alliance, the need for more internal marketing is an issue and something that is currently being addressed. Add: discomfort with new model; would have been good to have a general list for updates; remind people that this is s a pilot – need for flexibility; inability to answer questions right away, as answers are being worked out.Finally, one possible marketing success is that the project appears to be increasing the acquisition of ebooks within the Alliance at the individual library level. A number of libraries with little or no previous ebook experience have started to purchase ebooks since the implementation phase of the project began.
• #24: 2 sessions were held soon after the project was launched in July, in two central locations. Additionally, training materials were made available on-line for those who could not attend the in-person sessions. Additionally, an annual Alliance meeting provided the opportunity for an additional training and informational session with members of the CDMC, library directors and others from the Alliance. Based on survey results, the training sessions were well received, overall. That said, it was clear, especially during the general Alliance meeting, that communication channels had not been working as planned given the number of questions regarding the basic structure and workflow of the project. The group received questions that had been covered in the implementation team’s FAQ and email updates to the liaison group. While the training sessions were well received, there was some concern about the timing of them. Due to the workflow issues that arose late in the implementation process, the scheduling of these sessions was delayed a few times. In hindsight, it is clear that explaining the reasons for the delays at the time might have allayed concerns.
• #26: Why so much emphasis on evaluation? With thirty-six libraries involved in the e-book pilot, we are dealing with a mix of funding levels and a mix of comfort levels with this type of experiment. So by emphasizing evaluation, we hope to definitively show return-on-investment for those libraries who felt uneasy about their level of contribution as compared to their small available budget. We also want to set the stage for a continuing program, past the six-month pilot. If we can show usage from all the libraries and show that their users received a measurable benefit from the program, we will be better equipped to create a continuing program that involves the whole group. Additionally, to make this continuing program work, we are evaluating the structure of the pilot as it progresses to understand which components are successful and which will need to change.Another factor in evaluation is the publisher involvement. We need to demonstrate the value of the model to the publishers involved, as well as to those publishers who decided they did not want to contribute content. If the pilot shows a good outcome for the publishers involved, more may be convinced to join.Finally, there is the broader impact. We are the first group of this size to attempt a demand-driven pilot and many other consortia are interested in our progress. We have received calls and emails weekly from consortia interested in the pilot. Creating a detailed and meaningful evaluation will help other groups structure their programs to achieve their goals.
• #27: An early lesson learned was that monthly expenditure reports posted on our consortium website would not be nearly enough for the thirty-six member libraries. Instead, we moved to weekly reports emailed to a list of liaisons, who were then asked to share as needed with their colleagues. We brought these together into monthly reports that were emailed and posted on the website. These monthly reports expanded the weekly information to provide usage information for each library. Finally, EBL set up a real-time reporting site for the use of the implementation team. The team could access this site at any time to get current information about the pilot usage and expenditures, in order to troubleshoot or answer unexpected questions.
• #28: For a six-month pilot, we expected to do a midpoint evaluation at three months, only to realize that we had spent far less than half the money at that point. So what really is the midpoint then? Half the time or half the money?We realized, though, that the midpoint evaluations didn’t depend on dividing the time or money exactly in half. We planned to survey the thirty-six libraries to understand the effectiveness of our training and that needed to be done before the training faded from memory. We had also planned to create usage case studies of a few libraries to show examples of the effect of the program. However, we decided that, even if we included directions on how to calculate return-on-investment, not all the libraries would go through the process, so we created profiles of all thirty-six, instead of a few case studies. We also made the calculation method available in case libraries want to do a later analysis.
• #29: For a six-month pilot, we expected to do a midpoint evaluation at three months, only to realize that we had spent far less than half the money at that point. So what really is the midpoint then? Half the time or half the money?We realized, though, that the midpoint evaluations didn’t depend on dividing the time or money exactly in half. We planned to survey the thirty-six libraries to understand the effectiveness of our training and that needed to be done before the training faded from memory. We had also planned to create usage case studies of a few libraries to show examples of the effect of the program. However, we decided that, even if we included directions on how to calculate return-on-investment, not all the libraries would go through the process, so we created profiles of all thirty-six, instead of a few case studies. We also made the calculation method available in case libraries want to do a later analysis.
• #30: We are planning the final evaluation for February 2012. We’ll start with another survey to learn how the member libraries felt about the process of loading MARC records locally, the profile of titles, the publisher list, and the perceived effect of discovery option choice. We will look if the quality of cataloging affected usage – i.e. if better-cataloged titles were more likely to be purchased. Finally, we’ll examine usage by a number of different measures:Content:Usage by subject and publisherCollaboration and Dissemination:Number of unique users overall and per library, and how many schools contributed to the purchasesDiscovery:Usage based on discovery options and timing of record loadReturn on Investment:List price of content accessed/loaned/purchased versus individual library contribution By creating this final report, we hope to show the value of the program to the Alliance libraries and to the contributing publishers. We also hope to understand how we can evolve the program in the future to best meet the needs of our group.