Cloud Computing and Archives

Editor's Notes

• #2: Take home message - there really isn't anything to fear about the cloud… it's a technology which Information professionals (archives/RM/GLAM) should harness Please feel free to interrupt and ask questions!
• #3: - PA provides a RM and archives service for both Houses of Parliament. Preserves and provides public access to the records, no matter what form they take. The Act Room is the most visual representation of our collection (64,000 parchment rolls). Obvious national importance… - … Digital records may not be as physically stimulating, but they remain of equal value. - My role: Senior Digital Archivist, part of the P&A team. Role dedicated to DigiPres. - DigiPres in Parliament: Roots in 2008 with previous Director Caroline Shenton recognising the need to act - Adrian Brown on board in 2009. Concentrating on gaining buy-in and business case… - … for DPP, 2010-2015. Established current digital preservation capability. Implemented digital repository (Preservica EE, formerly SDB). Much more than just technology! A lot of work and effort gone into workflows/processes/advocacy… … 2015-present day. "Business-as-Usual". What does (buzzword) BAU actually mean? I'm still finding out in DigiPres terms… as is everyone.
• #4: - You may assume I know what I'm talking about… I'll share a secret with you… your guess is as good as mine when it comes to Digital Curation and Preservation best practices. - There may be different levels of experience, but everyone is learning, it's part of the joys of the job. It's certainly never dull! - I have felt like this countless times… Although the dog seems to have a pretty good grasp of what he's doing. - Also applies to how many archivists feel when confronted with cloud computing. - On a more positive note - Parliamentary Archives case study shows it can be done with the right considerations.
• #5: - There's a lot of assumptions and myths to do with Cloud technology. Some of which will be covered… - "The cloud" is such a ubiquitous term. Especially across the current digital landscape. - Paul Miller; "Cloud Computing promises — and often delivers — increased agility, and a means to bootstrap both startups and new projects inside established organisations. Cloud Computing and the emergence of data as “the new oil” are proving powerfully transformative in much that we do. Together, their potential to empower and disrupt is only just becoming apparent." - Grand aims indeed. Cut through the hyperbolic rhetoric and essentially cloud computing is "is a kind of internet-based computing, where shared resources and information are provided to computers and other devices on-demand." [Wikipedia]. - It's no different to using someone else's computing power to do your dirty work.
• #6: - Term originates as far back as 1996 - concept of distributed computing older than you think. - Everyone uses the cloud! - Social media/banking/communication/applications/servers/computing resource etc. etc. - Going back to ubiquitous nature of "THE CLOUD" - Archivists shouldn't be afraid to embracing change and opportunities which come with cloud computing.
• #7: - PA an early adopter of cloud services for the purposes of digital preservation. - I'll try and explain the rationale for doing so, and how we have addressed the various challenges and opportunities arising from this approach. - We conclude that the cloud can form part of a sustainable digital preservation service, provided the risks are correctly identified and appropriately mitigated. - The decision to use the cloud in this case must be understood in the context of a wider drive of using cloud services within the UK public sector, and the development of the G-Cloud Framework. - Parliament operates a 'Cloud-First' ICT policy. If a service is able to use the cloud, then the cloud must be considered as an option. PA actually the first case study of this policy in action in Parliament.
• #8: - Part of the digital repository infrastructure. While it was decided that the digital repository management system should be managed in-house, the storage platform for the repository content was identified as a candidate for using the cloud. - In the centre you have the Digital Repository / also a mirrored public instance. Synchronised with CALM cataloguing software, with links to hosted landing pages to publically available content. - Most important thing to note is that open content (anything which is public domain) stored with cloud storage providers / sensitive closed content stored on an internal storage platform. - Copies of all content mirrored between two different cloud service providers. The two cloud storage services operate on different technology platforms, one being based on EMC Atmos, the other on Amazon S3 Web Services.
• #9: When think about cloud storage thinking is often very negative… Key concern was to ensure Parliament’s ability to comply with relevant legislation (FOI, DPA), as well as addressing data sovereignty. Addressed quite straightforwardly by ensuring that the relevant requirements were clearly defined and subsequently incorporated into contracts. Each contract clearly states the respective roles and obligations of Parliament and the service provider with respect to legal compliance and data sovereignty. Specific requirement that all data must be hosted entirely within the European Economic Area, to ensure that it is covered by European privacy legislation and not subject to other jurisdictions, and hence to legislation such as the US Patriot Act. Information security: inherently lessened by the decision to only use the cloud for storing open data which is already in the public domain. Framing our requirements, we were also able to invoke relevant UK - Government information security standards (through G-Cloud accreditation). - A question of trust: rather than technical issues – there is no inherent reason for cloud providers to be less secure than inhouse infrastructure. - Authenticity: In the context of storage, durability is the most significant aspect of authenticity to consider. - A certain degree of loss must be anticipated, given a large enough volume of data stored over a long enough time period. - Dave Rosenthal (DHSR blog) a great source to consider for further arguments for and against cloud storage. - Determining the durability of a storage system is problematic (e.g. Amazon 99. power of 9). Difficult to discover scientific basis… - Durability problem not unique to the cloud. Archives have always had to manage a certain degree of loss. Our job is to reasonably minimise risk. Loss cannot be eliminated altogether.
• #10: - Challenges remain… - Economics: Operate PaYG model. Different to traditional IT model. Requires culture change… - Risk of high cost of downloading/retrieve content. - Portability: Primarily an issue when moving from one service provider to another. - The data volumes (TB's, not PB's) which PA expects to store within the next 5 years, while comparatively small compared to many memory institutions, would nonetheless be challenging to move. - Dependent on technology catching up. Managing risks through two suppliers which are staggered.
• #11: Elasticity: digital volume increasing = cloud storage becomes very appealing. Can be easily scaled to meet demand. - Characteristics of the cloud / low entry barrier / cost / flexability - Digital preservation requires constant and proactive management, irrespective of whether it takes place within the cloud or some other environment.
• #13: - In many ways, cloud storage has proven very straightforward to incorporate within the repository infrastructure. - The main infrastructure challenges have arisen from Parliament's ICT network (e.g. bandwith/storage adaptor proxy settings). - Large data networks = dedicated servers and internet connections. - All challenges have proved manageable. - One specific element of the digital repository infrastructure, is in many ways little different to providing that element in-house. - Risks arising from the cloud are often similar to those pertaining to other storage technologies. - Delegation of certain responsibilities to a third party.
• #14: - Cloud storage one specific element. - Remember cloud computing can apply to a number of scenarios - this is what I want to begin to explore now… - Think outside the box. Use the opportunities provided by cloud technology to meet current challenges.
• #15: - Approaches have the potential to transform preservation. - But the cloud is not simply about storage, providing new capabilities too, enabling more flexible responses to the preservation challenge. - Migration services / metadata creation / characterisation / integrity checking - Migration vs Emulation preservation strategies a long running argument / cloud services has the ability to transform this argument. - Migration has been preferred option for 10 years, but I would argue there are limited case studies and limited impact
• #16: Emerging service model based on cloud technologies Range from specific infrastructure elements (e.g. storage) to digital preservation “solutions” (e.g. Preservica)
• #17: Archivematica - Open Source / Service offered to member institutions. Community based digital preservation model. Typical example of DIgiPres services not being available at a local level. Grassroots development led to preservation storage and solutions being developed. Cloud hosting service (dubbed EduCloud) developed by the University IT centre, but other options do exist (Arkivum, DuraCloud). For participating members benefits of open source shared - lower costs - pool resources - distributed support For Archivmatica benefits include low level issues are manged - easy deployment of solution - cost of support contract reduced Artefactual can provide training - planning - operational service Cost efforts and support models can act as a template for other initiatives Can be tricky to start up and find like-minded institutions (Estates experience).
• #18: Emulation still seen as a highly complex and resource intensive preservation strategy. However, it offers potentially huge benefits. Ideal for complex objects (BIM in Parliament?) Allows end users to interact in the original enviroments. One of the most high profile success stories for emulation in digital preservation is the Internet Archive Console Living Room (1000s of games preserved and accessible in-browser) Cloud technology key to realising benefits of emulation.
• #19: University of Freiburg - Emulation as a Service (DPC Award Winner). Access historic digital documents and software in their native environments Re-enact business processes and understand past decisions Big barrier to this approach is IPR. IRP of applications/systems etc.
• #20: Frustration I have between Theory and Practical. Digital Preservation can often be full of theory, and less of the practical. PA work grounded deeply in practical. Theory plays a part in understanding possibilities for application of cloud technologies and the implications involved. Practical work allows the sustainable implementation of a solution.
• #21: Does the cloud make it easier to engage in digital preservation - inconclusive. Some success, but hardly a transformation… Can we ever truly trust could services? Can the cloud help core DP challenges? Storage - YES / Costs - Maybe not / Skills - yes in a narrow sense / Large scale computing - YES / Affordable emulation - getting there My own mantra / Keep things simple and practical