Importance of data standards for large scale data integration in chemistry
Download as PPT, PDF0 likes2,124 views
The document discusses the significance of data standards in large-scale data integration within the field of chemistry, emphasizing the need for comprehensive adoption of standards like molfiles and jcamp for effective data sharing. It highlights the limitations of existing standards without vendor support and advocates for enhanced digital formats for spectra to improve accessibility and utility. Additionally, it calls for better education on these standards to facilitate their adoption among chemists and researchers.
Importance of data standards for large scale data integration in chemistry
- 1. Importance of data standards for large scale data integration in chemistry Antony Williams, Valery Tkachenko, Alexey Pshenichnov, Ken Karapetyan, Stuart Chalk, Daniel Lowe and Carlos Coba ACS Denver, March 2015
- 2. Free and Easy • To make it easy to “take notes” these slides will be available at: www.slideshare.net/AntonyWilliams/
- 4. Charles Holland Duell • 1898-1901: US Commissioner of Patents • "Everything that can be invented has been invented."
- 5. Antony John Williams (et al)
- 6. Antony John Williams (et al) • “We don’t need more standards!” • “Of COURSE we can build a spectral database!” • “The standards we have are good enough”
- 7. A Pragmatic View to Progress • Let’s consider progressing an NMR Spectral database for the community! • MUST HAVES– spectra (1D/2D), associated structures, assignments • WANTS – predict NMR spectra, spectral searching, privacy/embargos • What would we need in terms of standards? • Molfiles and JCAMP
- 9. Standards
- 10. 2D NMR
- 13. Standards without adoption are limited in value • If the instrument vendors don’t support or adopt the standards success is limited • YESTERDAY discussion about publishing NMR – JCAMP • But what is already available will work – Jeol, Bruker, Thermo, Anasazi, Agilent/Varian - imperfect but useful
- 15. 9400 Spectra and growing http://www.chemspider.com/spectra.aspx
- 18. JCAMP file downloads • When NMR spectra are stored as JCAMP then downloads into offline packages are feasible – MestreLabs, ACD/Labs etc • Open Data – download versus view • Store spectra locally and reuse • Java is increasingly a pain! • Need to move to HTML5 viewing on ChemSpider, especially for Mobile Viewing
- 19. Challenges with Spectra • JCAMP is good for a lot of spectral data – IR, Raman, 1D NMR • MS data is rarely made available in JCAMP • We would love a ratified JCAMP 6.0 for 2D data exchange – allows third parties to build support for download • ASSIGNED JCAMP spectra supported
- 20. Proper Verification 03/25/15 Advanced Chemistry Development, Inc. (ACD/Labs) 20
- 21. Jmol - JSpecView
- 23. Spectral Display in the hand
- 24. New Repository Architecture doi: 10.1007/s10822-014-9784-5
- 25. Compounds
- 26. Reactions
- 27. Analytical data
- 30. ESI – Text Spectra
- 31. Developing Proof-of-Concept • Extract from 1976-2014 USPTO applications *unknown – starts off with NMR: peak list (no nucleus) H 975543 C 56536 unknown 44306 F 9429 P 3241 B 91 Si 62 Sn 22 Se 11 N 8
- 32. We want to find text spectra? • We can find and index text spectra:13C NMR (CDCl3, 100 MHz): δ = 14.12 (CH3), 30.11 (CH, benzylic methane), 30.77 (CH, benzylic methane), 66.12 (CH2), 68.49 (CH2), 117.72, 118.19, 120.29, 122.67, 123.37, 125.69, 125.84, 129.03, 130.00, 130.53 (ArCH), 99.42, 123.60, 134.69, 139.23, 147.21, 147.61, 149.41, 152.62, 154.88 (ArC) • What would be better are spectral figures – and include assignments where possible!
- 34. 1H NMR (CDCl3, 400 MHz): δ = 2.57 (m, 4H, Me, C(5a)H), 4.24 (d, 1H, J = 4.8 Hz, C(11b)H), 4.35 (t, 1H, Jb = 10.8 Hz, C(6)H), 4.47 (m, 2H, C(5)H), 4.57 (dd, 1H, J = 2.8 Hz, C(6)H), 6.95 (d, 1H, J = 8.4 Hz, ArH), 7.18–7.94 (m, 11H, ArH)
- 35. 13C NMR (CDCl3, 100 MHz): δ = 14.12 (CH3), 30.11 (CH, benzylic methane), 30.77 (CH, benzylic methane), 66.12 (CH2), 68.49 (CH2), 117.72, 118.19, 120.29, 122.67, 123.37, 125.69, 125.84, 129.03, 130.00, 130.53 (ArCH), 99.42, 123.60, 134.69, 139.23, 147.21, 147.61, 149.41, 152.62, 154.88 (ArC)
- 36. ESI Data also contains figures
- 37. Publications & “Real Spectra” • We are turning text into spectra • We are turning figures into spectra
- 38. Early Test Experiments Input 74 supplementary data documents. 3444 pages Output Plot2Txt extracted content from 1069 pages 1151 spectra total - >80% of peaks extracted to within 1-2 decimal places (ppm)
- 39. “Where is the real data please?” FIGURE DATA
- 40. Manual Curation Layer • ALL SPECTRA WILL BE STORED AS JCAMP • ChemSpider has had a manual curation layer for >8 years • Users can annotate data on ChemSpider • We do receive useful feedback from the community on the data and are optimistic!
- 41. Extraction is the WRONG WAY • We should NOT mine data out – digital form! • Structures should be submitted “correctly” • Spectra should be digital spectral formats, not images • ESI should be RICH and interactive • Data should be open, available, with meta data and provenance
- 42. We can solve for Authors here Will it be used though??? YES!
- 43. Supplementary Info Data now..
- 44. Data mining – it’s MINE!!!
- 45. What should we be doing? • Settle on a short-term format – JCAMP-JMOL?
- 46. But there ARE solutions!
- 47. But there ARE solutions!
- 48. What should we be doing? • Settle on a short-term format – JCAMP-JMOL? • Convince the instrument vendors to export in this format • Push button depositions into “containers” – ChemSpider, NMRShiftDB, Institutional Repositories • Encourage format support in software (read and write) – Mestre, ACD/Labs, Bruker TopSpin, etc.
- 50. Standards in Large Scale Data Integration • ALL of these are imperfect standards • Molfiles • SDF • InChI • JCAMP • But what can be done with them?
- 51. Compound Data • The standards of chemical structure handling are primarily molfile, SDfile, SMILES, InChI • We primarily depend on molfiles and SDF files for data deposition and interchange • We use InChI a lot – especially for integrated searching across the web
- 52. Searching the Entire Web?
- 53. Searching Internet by Structure
- 54. Compound Data • The standards of chemical structure handling are primarily molfile, SDfile, SMILES, InChI • We primarily depend on molfiles and SDF files for data deposition and interchange • We use InChI a lot – especially for integrated searching across the web • There ARE data interchange problems associated with structures….
- 55. USE and TEACH Standards • Too few people are aware of the existing standards and their capabilities • Part of the CINF mission activities should be to teach standards and this is being done • Still too few people have heard of InChI and JCAMP for example • Still little known about the importance of correct structure representations – kudos to people like Leah et al who TEACH THIS!
- 56. USE and TEACH Standards!
- 57. USE and TEACH Standards!
- 58. CVSP: Validate and Standardize
- 59. CVSP Rules Sets
- 60. CVSP Filtering of DrugBank
- 61. Compounds
- 62. Reactions
- 63. Use Ontologies
- 65. Contribute to PUBLIC Ontologies • Yes there are “company” ontologies – but for the good of the community contribute to public ontologies and standards • For data interchange and meshing this is soooooo beneficial!
- 66. ChAMP – Stuart Chalk
- 67. Use standards in APIs, endpoints and widgets
- 68. Semanticize content : RDF
- 69. Actions • Support and encourage new standards • In the meantime, reawaken and modernize the JCAMP standard • Show up and listen to Bob Hanson today • Encourage scientists to provide data
- 70. Charles Holland Duell in 1902 “…all previous advances in the various lines of invention will appear totally insignificant when compared with those which the present century will witness. I almost wish that I might live my life over again to see the wonders which are at the threshold”
- 71. “Git-r-Done”
- 72. Acknowledgments • Daniel Lowe – NextMove, Reactions and Spectra • Bill Brouwer – Plot2Txt Development • Carlos Cobas and Stan Sykora– MestreLabs • The ChemSpider team – led by Richard Kidd • The RSC Data Repository team
- 73. Thank you Email: williamsa@rsc.org ORCID: 0000-0002-2668-4821 Twitter: @ChemConnector Personal Blog: www.chemconnector.com SLIDES: www.slideshare.net/AntonyWilliams
Editor's Notes
- #21: All lines up nicely.