Darwin Core Archive (DwC-A) validation: A New Collaborative Effort
- 1. Darwin Core Archive (DwC-A) validation: A New Collaborative Effort Christian Gendreau, UniversitĂ© de MontrĂ©al / Canadensys David P. Shorthouse, UniversitĂ© de MontrĂ©al / Canadensys Marie-Ălise Lecoq, GBIF France Tim Robertson, GBIF
- 2. Darwin Core Archive (DwC-A) DarwinCore standard does not impose strong rules on the content associated with any DarwinCore terms.
- 3. Current GBIF DwC-A Validator Original goal â⊠test Darwin Core Archives as specified in the Darwin Core Text Guide.â http://tools.gbif.org/dwca-validator/
- 4. Current GBIF DwC-A Validator Original target DwC-A are simple and can be created using simple custom scripts. â⊠make sure GBIF and others can read the information as expected.â
- 5. Current GBIF DwC-A Validator âą Validates archive structure âą Offer web presence â Report viewer â API
- 6. Next GBIF DwC-A Validator? New goal Extends validation to the content of the archive https://github.com/gbif/dwca-validator
- 7. Current content validators âą Atlas of Living Australia sandbox âą VertNet â Spatial quality âą GBIF Spain â Darwin Test âą Encyclopedia of Life â dwc-validator âą Scratchpads â dwca-validator âą GlobalNames â dwc-archive ruby gem ⹠⊠much more See Appendix 1 for links
- 8. What we need? âą Accommodate different scopes âą Configuration/customizations â Use more knowledge when available âą Web access (page and API)
- 9. Scopes âą Data entry âą Desktop software â Scientific Work Flow â Statistical software âą Integrated Publishing Toolkit (IPT) âą National nodes âą Aggregators
- 10. Configuration/Customization âą Where the validator will be used? âą Can we provide more information? â e.g. I know all the dates in my file should be ISO
- 11. Components âą Library âą Web âą Extension Support
- 12. Library âą Define structure for validation process âą Provide a validation framework enabling sharing âą Close to DarwinCore specification
- 13. Web âą Web page to submit archive or URL âą Report viewer âą API
- 14. Extension Support âą Include domain knowledge âą Propose interpreted data
- 15. Internals âą Validation types â Structure âą Metadata â Records : Rows âą Fields data (e.g. date, coordinates) â Records : Columns âą ID uniqueness
- 16. Internals â Record level âą Validation chain â Composed by chain elements â Possible parallelism
- 17. Internals â Record level âą Immutable Chain element â Self contained âą Never relies on another chain element â Ordering independent âą Same behaviour wherever the element is used in the chain But what if I need really ordering?
- 18. Internals - Composition âą Composed chain element âą Exposed as one chain element
- 19. Composition example âą Mandatory Latitude/Longitude â Check record completion on lat/long â Check decimal lat/long value
- 20. Configuration example âą Select mandatory DarwinCore terms â scientificName must be provided âą Restrict bounding box â decimalLatitude and decimalLongitude must be between
- 21. Customization example âą Apply your own controlled vocabulary â Use your own dictionary for a term â ControlledVocabularyEvaluationRule
- 22. Extension Example âą Suggester, link to narhwal-processor â SuĂšde â> ISO 3166-2:SE â URI â> http://sws.geonames.org/2661886
- 23. Collaborative âą Share configuration âą Share customization (dictionary) âą Implement new reusable component â e.g. validation on specific Dwc-A extension
- 24. Collaboration âą Where to go? â https://github.com/gbif/dwca-validator âą Who can contribute? â Everyone âą What is needed? â Ideas, constructive comments â Code review, feedback
- 25. Project status âą Not yet released âą Command line interface available Follow the project on GitHub
- 26. Acknowledgments
- 27. Special thanks âą SiB Colombia âą SiB Brazil âą Peter Desmet âą John Wieczorek âą Dag Endresen âą âŠ
- 28. Appendix 1 DwC Content validators Atlas of Living Australia sandbox http://sandbox.ala.org.au/datacheck/ VertNet â Spatial quality Displayed on occurrence pages at http://portal.vertnet.org/search GBIF Spain â Darwin Test http://www.gbif.es/darwin_test/Darwin_Test_in.php Encyclopedia of Life â dwc-validator http://services.eol.org/dwc_validator/
- 29. Appendix 1 - continue Scratchpads â dwca-validator https://github.com/edwbaker/dwca_validator/ GlobalNames â dwc-archive ruby gem https://github.com/GlobalNamesArchitecture/d wc-archive
Editor's Notes
- #18: explain immutable
- #21: ----- Notes de la réunion (2014-10-20 14:54) ----- examples
- #23: ----- Notes de la réunion (2014-10-20 14:54) ----- suggester : explain it
- #25: ----- Notes de la réunion (2014-10-20 14:54) ----- collaboration received where to go current state, timeline current challenges, collaboration is needed