The Rhetoric of Research Objects

Editor's Notes

• #2: We have all grown up with the research article and article collections (let’s call them libraries) as the prime means of scientific discourse. But research output is more than just the rhetorical narrative. The experimental methods, computational codes, data, algorithms, workflows, Standard Operating Procedures, samples and so on are the objects of research that enable reuse and reproduction of scientific experiments, and they too need to be examined and exchanged as research knowledge. We can think of “Research Objects” as different types and as packages all the components of an investigation. If we stop thinking of publishing papers and start thinking of releasing Research Objects (software), then scholar exchange is a new game: ROs and their content evolve; they are multi-authored and their authorship evolves; they are a mix of virtual and embedded, and so on. But first, some baby steps before we get carried away with a new vision of scholarly communication. Many journals (e.g. eLife, F1000, Elsevier) are just figuring out how to package together the supplementary materials of a paper. Data catalogues are figuring out how to virtually package multiple datasets scattered across many repositories to keep the integrated experimental context. Research Objects [1] (http://researchobject.org/) is a framework by which the many, nested and contributed components of research can be packaged together in a systematic way, and their context, provenance and relationships richly described. The brave new world of containerisation provides the containers and Linked Data provides the metadata framework for the container manifest construction and profiles. It’s not just theory, but also in practice with examples in Systems Biology modelling, Bioinformatics computational workflows, and Health Informatics data exchange. I’ll talk about why and how we got here, the framework and examples, and what we need to do. [1] Sean Bechhofer, Iain Buchan, David De Roure, Paolo Missier, John Ainsworth, Jiten Bhagat, Philip Couch, Don Cruickshank, Mark Delderfield, Ian Dunlop, Matthew Gamble, Danius Michaelides, Stuart Owen, David Newman, Shoaib Sufi, Carole Goble, Why linked data is not enough for scientists, In Future Generation Computer Systems, Volume 29, Issue 2, 2013, Pages 599-611, ISSN 0167-739X, https://doi.org/10.1016/j.future.2011.08.004
• #4: Analogy Emotional appeal
• #10: Linking, “Packaging” & Citing Codes, Data, Models, SOPs, Samples, Strains, Articles, People, Projects, ELNs….
• #11: Impacts on metadata and on transfer and access
• #14: ROs combine containers and incremental metadata
• #15: Mimetype: robundle+zip ZIP or BagIt folder structure JSON and YAML Linked-ISA 
• #17: By reference
• #18: Release like software http://www.timeshighereducation.co.uk/news/evolving-manuscripts-the-future-of-scientific-communication/2020200.article Evolving manuscripts’: the future of scientific communication? 14 May 2015 | By Holly Else Chief scientific adviser Sir Mark Walport posits a future in which papers are revised as research matures, supplanting ‘outmoded’ publishing practices Source: Universal/Kobal If you put your mind to it: new methods of publishing could change everything In years ahead, scientists may communicate their results through “evolving manuscripts” that are updated continually over a working life. This scenario was put forward by Sir Mark Walport, the government’s chief scientific adviser, at a conference on the future of publishing. Scientists could end up with three publications that span the whole of their career in such a system, which could end today’s “completely outmoded” publishing practices, he said. Sir Mark was speaking at the second part of the Royal Society’s Future of Scholarly Scientific Communication conference, held in London on 5-6 May. His idea would help to mitigate “perverse incentives” in the current system, he said. These include a bias against publishing negative results or those that confirm or confound existing research, as well as the pressures that scientists face to split a piece of work into multiple articles. “We must facilitate new ways of publication…We have hardly scratched the surface of the potential of new publishing models to communicate science in much better ways than we have been doing,” he said. An “evolving manuscript” would begin with a pre-publication, pre-peer review “beta 0.9” version of an article, followed by the approved published article itself, which Sir Mark dubs “version 1.0”. Subsequently, scientists would update this paper with details of further work as the area of research develops. Versions 2.0 and 3.0 might allow for the “accretion of confirmation [and] reputation”, for example. “The idea that you start from scratch with every [new] paper and you publish a bit of new data [so that] you slightly change the discussion is a completely outmoded way of doing science in the 21st century,” Sir Mark added. “One could have a much more organic publication that would include the repeats of the work that would publish automatically alongside it,” he explained. There would need to be a system to “Kitemark” an evolving manuscript and highlight the most up-to-date version, he said. A “golden thread” linking the body of work would also be necessary. “The thread would need to be rewritten on a continual basis,” he added. “We could be in a world where you write three papers in your entire life, and they just evolve,” he said. Sir Mark added that the system might encourage more debate among scientists about research after work has been published. Post-publication peer review so far has an “abysmal” record among scientists, he said. There is an “issue with the culture of science” that means researchers are “pretty good” at criticising each other at meetings and conferences but are “very, very bad” at being willing to criticise each other in post-publication peer review, he said. Ottoline Leyser, director of the Sainsbury Laboratory at the University of Cambridge, said this might be because at the moment the assessment criteria in science revolve around the individual. “People have stopped thinking about the scientific enterprise,” she said. “If someone criticises your work, that is a jolly good thing; and if it turns out you are wrong, that is excellent because then you have disproved your hypothesis and can move forward,” she added. Such issues are “fundamental to the philosophy of science” and to research progress. But they are now “associated with negative impact on people’s careers”, she explained. holly.else@tesglobal.com
• #20: Replace with a workflow? Fixivity - Liveness New/updated/deprecated methods, datasets, services, codes, h/w Snapshots Dependency – Containment Streams, non-portable data/software, 3rd party services, supercomputer access, licensing restrictions…. Locally contained and maintained External dependencies Transparency Blackboxes, proprietary software, manual steps Robustness Bounds of use Stochastics, non-deterministics, contexts
• #21: Like BCO domains
• #22: Come back to this later. Fast Healthcare Interoperability Resources (FHIR, pronounced "fire") is a draft standard describing data formats and elements (known as "resources") and an application programming interface (API) for exchanging electronic health records. The standard was created by the Health Level Seven International (HL7) health-care standards Enabling Precision Medicine via standard communication of NGS provenance, analysis, and results doi: https://doi.org/10.1101/191783
• #24: ROs combine containers and incremental metadata Like DataONE Packages Its all about the metadata
• #25: And many other solutions…. http://bioboxes.org
• #26: https://www.w3.org/TR/annotation-vocab/
• #30: JERM and DC Terms and so on in those nested metadata.rdf files - one per SEEK resource that is part of the investigation
• #32: Checklist checks what annotations should be there Some will be word docs Some will RDF Graphs extracted from, say CWL, “there has to be a CWL file” Turtles all the way down Currently CWL in a RO (with the files) A CWL RO has the parts exposed. Version and provenance in the RO model Checklists in the annotations
• #34: DataCatalog and Datasets are relevant
• #35: Mandatory
• #36: Revise Citation Library Experiment Bio specific
• #37: ShEx doesn’t do linking http://www.rohub.org/portal/ro?ro=http://sandbox.rohub.org/rodl/ROs/IPWV_Iceland/ no entries Its like a myExperiment Pack but with a checker http://www.rohub.org/portal/ro?ro=http://sandbox.rohub.org/rodl/ROs/HD_chromatin_analysis/
• #38: Use with JSON schema They are separate approaches. ShEx is a pattern language with its own syntax that looks kind of like SPARQL. SHACL on the other hand is expressed as rules in RDF, and is more similar to declarative languages like XSLT and Prolog as it has a fixed traversal pattern (which can be used to instance to build XML or JSON while it trundles along) Shape Expressions (ShEx) language describes RDF graph structures. These descriptions identify predicates and their associated cardinalities and datatypes. ShEx shapes can be used to communicate data structures associated with some process or interface, generate or validate data, or drive user interfaces. It is intended to: validate RDF documents. communicate expected graph patterns for interfaces. generate user interface forms and interface code. SHACL Shapes Constraint Language, a language for validating RDF graphs against a set of conditions. These conditions are provided as shapes and other constructs expressed in the form of an RDF graph. RDF graphs that are used in this manner are called "shapes graphs" in SHACL and the RDF graphs that are validated against a shapes graph are called "data graphs". As SHACL shape graphs are used to validate that data graphs satisfy a set of conditions they can also be viewed as a description of the data graphs that do satisfy these conditions. Such descriptions may be used for a variety of purposes beside validation, including user interface building, code generation and data integration.
• #39: Now while the BCO references these resources in several places in its JSON structure, some may also be indirectly referenced. For instance the CWL workflow might reference particular Docker images that capture the Python version to use. W3C PROV files might be provided, which can explain more detailed provenance of workflows; this might however become specific to the workflow engine used, and might not be directly identified all the resources seen in the BCO. While we can identify authors with ORCID, they might author different parts of the BCO. If you made a clever Python script used by a BCO, then it is only FAIR that you should be attributed – even if you were nowhere in the vicinity when the BCO was later created.So you can think of these pink, green and blue arrows here as each giving partial picture of what is the whole BioCompute Objects. There is also the question of how to move the BCO around – the JSON has many external references as well as relative references to plain files – how can you capture it all without understanding all of the BCO spec? We are looking at using the BagIt Research Objects for this purpose. Bag-It is a digital archive format used by Library of Congress and digital repositories. It handles checksums and completeness of files, even if they are large or external. Research Object (RO) is a model for capturing and describing research outputs; embedding data, executables, publications, metadata, provenance and annotations. Although it is a general model, ROs have been used in particular for capturing reproducible workflows. The combination of these, ro-bagit has recently used by the NIH-funded Big Data for Discovery Service for transferring and archiving very large HTS datasets in a location-independent way, so naturally this could be a good choice for how to archive BCOs. So here the manifest of the Research Object, ties everything together. The manifest is in JSON-LD format – so it is Linked Data – but you don’t have to know unless you really want to – it is also just JSON. The manifest **aggregates** all the other resources, including the BCO, but also external resources as well as outside references like identifiers.org. The aggregation also provide attribution and provenance of each resource, so they get the credit they deserve. This is of course also important for regulatory purposes, e.g. to check if the latest version of a tool was used. An important aspect of research objects is also to capture annotations, using the W3C Web Annotation Model. This allows any part of the BCO to be further described; textually or semantically; so you are not limited to what is supported by the specification of BCO or Research Object. In particular this might be where community-driven standards like BioSchemas can be used.
• #41: it would be the same wherever the git commit lives. So the links can also be generated locally with a git checkout - e.g. as we're doing in the cwltool reference runner provenance when we need to refer to what workflow was run solved the problem we had in Taverna where we didn't know where the workflow lived we still might not know that.. but if it's a public workflow and it later is visualized, then CWL Viewer can show it future-proof!
• #43: Issues of ShEx Should link follow Implementation of validators Can’t use off the shelf validators Because we can’t iterate over whats in the RO
• #44: “if we can get it into a couple other impls, we could have it in shex 2.1”
• #45: Acquired Look up URI (e.g. ORCID to author name) Add manually in UI, saved as independent annotations (Curator gets credit, does not touch other parts of RO) Not in the RDF Custom post-processing, extract/convert from domain-specific formats Use existing schema.org annotations Custom namespaces Post-processing, adding derived annotations (e.g. SPARQL CREATE) Acquired through e.g. URI look-ups Look up URI (e.g. ORCID to author name) Added through interfaces Embedded in nested annotation resources RDF Shape that indicates to follow links RO pre-processing that merges to single graph
• #48: for transferring and archiving very large HTS datasets in a location-independent way,
• #50: dataCrate – BagIt + Schema.org CWL – complex types https://github.com/UTS-eResearch/datacrate/blob/master/spec/0.1/data_crate_specification_v0.1.md they just mentioned https://github.com/UTS-eResearch/datacrate/blob/master/spec/0.1/data_crate_specification_v0.1.md on the public-scholarly-html list - a kind of bagit data profile with some schema.org JSON-LD annotations -- a bit similar to our https://w3id.org/ro/bagit and obviously a clear trend http://eresearch.uws.edu.au/blog/2013/11/01/introducing-next-years-model-the-data-crate-applied-standards-for-data-set-packaging/ https://github.com/ResearchObject/bagit-ro Research Object BagIt archive Document identifier: https://w3id.org/ro/bagit Author: Stian Soiland-Reyes http://orcid.org/0000-0001-9842-9718 BagIt is an Internet Draft that specifies a file system structure for transferring and archiving a collection of files, including their checksums and brief metadata. Research Object bundles is a specification for a structured ZIP-file, based on the ePub and Adobe UCF specifications. The bundle serializes a Research Object, embedding some or all of its resources within the ZIP file, and list the RO content in a manifest, in addition to embedding and referencing annotations and provenance. A BagIt bag can be considered a mechanism for serialization and transport consistency, while a Research Object can be considered a way to capture identity, annotations and provenance of the resources. As such, the two formats complement each-other. They are however not directly compatible. This GitHub repository explains by example a profile for a BagIt bag to also be a Research Object. Feel free to provide comments and raise issues, or suggest changes as pull requests. https://nightly.science.ai/documentation/archive Authoring platform Reproducible Document Exchange Format – to present online, and preserve as publisher An example published article, with enhanced reproducible version Announcement: https://elifesciences.org/for-the-press/e6038800/elife-supports-development-of-open-technology-stack-for-publishing-reproducible-manuscripts-online About the project: https://elifesciences.org/labs/7dbeb390/reproducible-document-stack-supporting-the-next-generation-research-article June 2017 survey results: https://elifesciences.org/inside-elife/e832444e/innovation-understanding-the-demand-for-reproducible-research-articles
• #51: 6427 views
• #52: Specialist, bespoke Rise of containers
• #54: The convergence b/w transitive credit to data and SW: credit to data requires provenance of the form "A used X and generated Y" so when Y gets recognition, X gets part of the credit, mediated by A (the transformation) but A is in fact a piece of SW which also belongs to someone so when Y gets recognition, part of it should go to A's contributors in addition to X's contributors internally, this credit to A may be distributed according to the dependency structure of A, i.e. some if it will go to the contributors of the libs that are used in A, for example so there is an external flow of credit from Y back to X through A so this is a combined data/SW credit model, and the portion that goes to A then originates a SW-only credit flow within A, that goes back to the dependencies used by A and gives credit to their contributors