Data Archiving and Sharing
Download as PPTX, PDF1 like2,210 views
The document discusses best practices for data archiving and sharing, emphasizing the importance of preserving data using appropriate file formats, creating metadata, and adhering to FAIR principles. It includes exercises to identify bad data management decisions and encourages researchers to utilize standards and repositories for better data accessibility and usability. Overall, it highlights the need for proactive data management throughout the research cycle.
Data Archiving and Sharing
- 1. Data Archiving and Sharing C. Tobin Magle, PhD July 25, 2017 10:00-11:00 a.m. Morgan Library Computer Classroom 175
- 2. Hypothesis Raw data Experimental design Tidy Data ResultsArticle Data Management Plans Cleaning Sharing Analysis Open Data Code Reproducible Research Reuse Closed Data Archiving The research cycle
- 3. Outline • Preserve: File formats and storage • Describe: Metadata standards and standard languages • Share: FAIR principles and repositories
- 4. Why even bother? Data Sharing and Management Snafu in 3 Short Acts: A data management horror story by Karen Hanson, Alisa Surkis and Karen Yacobucci. http://www.youtube.com/watch?v=N2zK3sAtr-4
- 5. Exercise 1: Worst practices • Make a list of bad data management decisions the researcher made at the end of his project. • Keep an eye out for what he could have done better during this presentation.
- 6. Preserve Backup, archival formats, description
- 7. Digital data preservation Short Term • During the project • Frequent changes • Your responsibility Long term • After the project is over • Little to no changes • Can be outsourced
- 8. Preservation best practices • Back up your data! • Save in archival formats • Include metadata
- 9. Data backup • Make 3 copies • Protects against natural disasters • Example • Computer HD • External hard drive • Cloud
- 10. Data formats • Avoid proprietary formats • Use common data standards in your field • Find standards: https://fairsharing.org/standar ds/?q=&selected_facets=type _exact:model/format
- 11. Proprietary formats and alternatives Proprietary Format Alternative Format Excel (.xlsx) Comma Separated Values (.csv) Word (.docx) plain text (.txt) or PDF/A (.pdf) PowerPoint (.pptx) PDF/A (.pdf) Photoshop (.psd) TIFF (.tif, .tiff) Quicktime (.mov) MPEG-4 (.mp4) MPEG 4 (.m4p) .mp3 (compressed)/.wav (uncompressed) https://www.loc.gov/preservation/digital/formats/content/content_categories.shtml
- 12. Excel guidelines • One table per sheet • Sheet -> .csv • No formatting • No images • No formulas • Make your data tidy
- 13. Excel Archival Tool • Input: Excel File • Output (per tab): • One .csv • One .txt for formulas • One HTML visualization • https://github.com/mcgrory/ ExcelArchivalTool
- 14. Exercise 2: Formats Think of one type of data that you produce: • What format is it in? • Is it proprietary? If so, what alternative format? • Is it the most commonly used data format for that type of data in your field?
- 15. Describe Metadata (README files, codebooks, Metadata standards)
- 16. Metadata • Relevant information for discovery, re-creation and re-use • Descriptive – using data • Discovery – finding data http://library.umassmed.edu/necdmc/necdmc_module3.pptx
- 17. Descriptive Metadata • Provides context: everything you need to know to interpret and reuse the data • Examples • Readme files • Code books http://library.umassmed.edu/necdmc/necdmc_module3.pptx
- 18. README files • Describe the contents of data files • List software necessary to interpret the data • Unstructured format: • (+) human readable • (-) not machine readable
- 19. Codebooks • Define the variables and their units • Explain the formats for dates, time, geographic coordinates • Define any coded values and missing values
- 20. Discipline specific metadata • Specify pieces of information to include • Specify format • Not available in all fields • Find standards • http://www.dcc.ac.uk//resources/metadata-standards • https://fairsharing.org/standards/?q=&selected_facets=type_exact:reporting% 20guideline
- 21. Exercise 3: • Think of the data you chose to work with in Exercise 2 • What information would you include a README file? A codebook? • Is there a metadata standard you could use?
- 22. Share FAIR principles, repositories, discovery metadata
- 23. FAIR principles • Findable: searchable and has a unique ID • Accessible: in a repository • Interoperable: Described in common standards for your field • Reusable: Properly described and licensed for reuse https://www.nature.com/articles/sdata201618
- 25. Licensing • State your conditions for reuse • Citation? • Creative common licenses are a good starting point • CC-0 for data
- 26. Interoperable Reusable Controlled Vocabularies Licenses File formatsMetadata Descriptive Metadata Codebooks Metadata standards Readme Community Standards Non-proprietary
- 27. Controlled Vocabularies • “Official” names for things • Ontologies: include relationships between terms • Search for relevant ontologies: https://fairsharing.org/standards/ ?q=&selected_facets=type_exa ct:terminology%20artifact https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/
- 28. Exercise 4: • Think about the data from exercises • Is there a standard ontology you could use to describe the data?
- 29. Accessible Interoperable Reusable Controlled Vocabularies Repositories Licenses File formats Persistent identifiers Metadata Descriptive Metadata Codebooks Web interface Metadata standards Readme Community Standards Non-proprietary
- 30. Repositories (aka databases) provide • A place to put (meta)data • Unique IDs for each dataset • A search interface • Metadata requirements
- 31. Finding Research Data Repositories FAIRSharing https://fairsharing.org/database s/ Registry of Research Data Repositories http://www.re3data.org/
- 32. CSU Digital Repository • Over 100 Datasets • Dublin core metadata • Supports all* (meta) data types • At no cost <1 TB • $150/TB for 5 years • $300/TB for >5 years *that we know of
- 33. CSU Repository Deposit Steps • Contact Tobin! – self deposit is in the works • Prepare: • Data files • Metadata (Readme.txt, codebooks, etc) • Any related additional files (like a license) • Deposit agreement: • Declare you have right to deposit • Give permission to repository to perform preservation and access procedures • Upload Module 7: Archiving & Preservation
- 34. Stable identifiers • URLs break • Stable identifiers are permanent in a database • Some provide linking capabilities • DOI – https://doi.org/10.1109/5.771073 • Handle- http://hdl.handle.net/10217/177356
- 35. Findable Accessible Interoperable Reusable Controlled Vocabularies Repositories Licenses File formats Persistent identifiers Metadata Descriptive Metadata Discovery Metadata Codebooks Search Interface Web interface Metadata standards Readme Community Standards Non-proprietary
- 36. Discovery Metadata • Make your datable findable • Metadata standards • Defined by the repository http://library.umassmed.edu/necdmc/necdmc_module3.pptx
- 37. Metadata standard: Dublin Core • Can be applied to anything • 15 core Elements: • http://dublincore.org/documents/dc es/ • CSU librarians help you write this metadata • Example: http://hdl.handle.net/10217/1802 80
- 38. Exercise 5: • How FAIR is your data? • Findable? • Accessible? • Interoperable? • Reusable?
- 39. Summary • Most of the work for sharing is part of Preservation - Think about it BEFORE you start your project (DMP) - Do the work as you go • Make preservation and sharing easier with a trusted repository • It’s complicated! Ask for help
- 40. Need help? • Email: tobin.magle@colostate.edu • Data Management Services website: http://lib.colostate.edu/services/data-management • File Format Quick Guide: https://en.wikibooks.org/wiki/Choosing_The_Right_File_Format/Quick_Guide • Excel Archival Tool: https://github.com/mcgrory/ExcelArchivalTool • DSpace Institutional Repository: https://dspace.library.colostate.edu/