![summer school
xmlsummerschool.com Slide 2
Licensed under a Creative Commons
Attribution-Noncommercial-Share Alike 4.0 International License
Citizen OpenNoteBook science
World citizens spend 500 Billion USD on STEM[1] research.
85% is wasted
Most citizens don’t have access
We are facing existential threats
• Climate
• Antibiotic resistance
• Biodiversity loss
The scientific literature may already contain answers
Let’s look for them…
OpenNoteBook is doing Everything on the web Immediately
[1] Scientific Technical Engineering Medical](https://image.slidesharecdn.com/xmlss19petermr-190918084149/85/XML-for-science-its-huge-potential-but-are-pubiishers-preventing-it-2-320.jpg)
![summer school
xmlsummerschool.com Slide 7
Licensed under a Creative Commons
Attribution-Noncommercial-Share Alike 4.0 International License
http://www.budapestopenaccessinitiative.org/read
… an unprecedented public good. …
… completely free and unrestricted access to [peer-
reviewed literature] by all scientists, scholars, teachers,
students, and other curious minds. …
…Removing access barriers to this literature will accelerate
research, enrich education, share the learning of the rich
with the poor and the poor with the rich, make this
literature as useful as it can be, and lay the foundation for
uniting humanity in a common intellectual conversation
and quest for knowledge.
(Budapest Open Access Initiative, 2003)](https://image.slidesharecdn.com/xmlss19petermr-190918084149/85/XML-for-science-its-huge-potential-but-are-pubiishers-preventing-it-7-320.jpg)
![summer school
xmlsummerschool.com Slide 8
Licensed under a Creative Commons
Attribution-Noncommercial-Share Alike 4.0 International License
[1] The Military-Industrial-Academic complex (1961)
(Dwight D Eisenhower, US President)
Publishers Academia
Glory+?
$$, MS
review
Taxpayer
Student
Researcher
$$ $$
in-kind
The Publisher-Academic complex[1]
Infrastructure
“The scholarly poor”](https://image.slidesharecdn.com/xmlss19petermr-190918084149/85/XML-for-science-its-huge-potential-but-are-pubiishers-preventing-it-8-320.jpg)
![summer school
xmlsummerschool.com Slide 16
Licensed under a Creative Commons
Attribution-Noncommercial-Share Alike 4.0 International License
PART of ContentMine stylesheet for JATS2HTML
<xsl:template match=“jats:article">
<html:head>
<html:style type="text/css” >
a {background : #ffffff;}
article {border-style : dotted;}
</html:style>
</html:head>
<html:body> <!-- PULL for main sections -->
<xsl:apply-templates select=”jats:front"/>
<xsl:apply-templates select=”jats:body"/>
<xsl:apply-templates select=”jats:back"/>
<!-- PUSH for unknown elements -->
<xsl:for-each select="*[not(
self::jats:front | self::jats:body | self::jats:back)]">
<xsl:call-template name=“unknownElement”/>
</xsl:for-each>
</html:body>
</xsl:template>
NOTE: JATS has ca 250 elements, not always used wisely](https://image.slidesharecdn.com/xmlss19petermr-190918084149/85/XML-for-science-its-huge-potential-but-are-pubiishers-preventing-it-16-320.jpg)
![summer school
xmlsummerschool.com Slide 22
Licensed under a Creative Commons
Attribution-Noncommercial-Share Alike 4.0 International License
ContentMine XML Search results
<snippetsTree> <!-- CProject -->
<snippets
file="oil186/PMC5080681/results/species/binomial/results.xml"
> <!-- Ctree -->
<result
pre="ify the chemical composition, and to assess
anthelmintic, antimicrobial and antioxidant effects of "
exact="Thymus bovei”
xpath="/html[1]/body[1]/div[2]/div[5]/p[1]" <!-- locates
snippet -->
match="Thymus bovei"
post=" essential oil. "
name="binomial"/>
…
NOTE: Each snippet with
• semantic origin “species/binomial/” links to dictionary
• separate XML result element
• W3C annotations (pre, exact, post)
• XPath locates it in document (W3C Annotations Rec.)](https://image.slidesharecdn.com/xmlss19petermr-190918084149/85/XML-for-science-its-huge-potential-but-are-pubiishers-preventing-it-22-320.jpg)
XML for science; its huge potential; but are pubiishers preventing it?
- 1. xmlsummerschool.com Slide 1 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 Unported License summer school Is “Publishing destroying science?” XML: Liberating science for citizens Peter Murray-Rust ContentMine.org XMLSummerSchool Trends and Transients September 2019
- 2. summer school xmlsummerschool.com Slide 2 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Citizen OpenNoteBook science World citizens spend 500 Billion USD on STEM[1] research. 85% is wasted Most citizens don’t have access We are facing existential threats • Climate • Antibiotic resistance • Biodiversity loss The scientific literature may already contain answers Let’s look for them… OpenNoteBook is doing Everything on the web Immediately [1] Scientific Technical Engineering Medical
- 3. summer school xmlsummerschool.com Slide 3 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Thanks and Volunteers • Ambarish Kumar (New Delhi, IN) • Daniel Nuest (WWU, DE) • Simon Worthington (TiB, DE) • Tiago Lubiana (USP, BR) • Emanuel Faria (Brasilia) • Graham Klyne (Oxford, UK) Peter Flynn (UCC, IE)
- 4. summer school xmlsummerschool.com Slide 4 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Resources Resources for XMLSS19 • http://github.com/petermr/xmlopensci • http://github.com/petermr/CEVOpen • http://github.com/petermr/climate • http://github.com/petermr/opennotebook • http://github.com/petermr/ami3 • Contains code, resources and Issues (including conversations) • https://media.ed.ac.uk/media/1_46h85ltt (Disrupting the Publisher-Academic Complex) • Scientific search for Everyone (https://www.slideshare.net/petermurrayrust/scientific-search-for-everyone )
- 5. summer school xmlsummerschool.com Slide 5 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Trends: Good and Ungood Existential threats (climate, disease) Good • Open science and Open Access • Free/Open Source • Distributed collaboration • Github • Wikidata, Linked Open Data and universal identifiers Ungood • Publisher-Academic Complex • North-South divide (knowledge neocolonialism)
- 6. summer school xmlsummerschool.com Slide 6 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Politics of Publishing
- 7. summer school xmlsummerschool.com Slide 7 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License http://www.budapestopenaccessinitiative.org/read … an unprecedented public good. … … completely free and unrestricted access to [peer- reviewed literature] by all scientists, scholars, teachers, students, and other curious minds. … …Removing access barriers to this literature will accelerate research, enrich education, share the learning of the rich with the poor and the poor with the rich, make this literature as useful as it can be, and lay the foundation for uniting humanity in a common intellectual conversation and quest for knowledge. (Budapest Open Access Initiative, 2003)
- 8. summer school xmlsummerschool.com Slide 8 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License [1] The Military-Industrial-Academic complex (1961) (Dwight D Eisenhower, US President) Publishers Academia Glory+? $$, MS review Taxpayer Student Researcher $$ $$ in-kind The Publisher-Academic complex[1] Infrastructure “The scholarly poor”
- 9. summer school xmlsummerschool.com Slide 9 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License
- 10. summer school xmlsummerschool.com Slide 10 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Demo: Plants, oils, medicines
- 11. summer school xmlsummerschool.com Slide 11 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Citizen OpenNoteBook science About 2 million STEM articles per year (excl. theses, reports) 70% behind paywalls Most in PDF (arghhh!) BUT… NLM (US NIH) requires its output to be Open and XML So we’re going to read a few million articles. Start at http://europepmc.org And use http://ContentMine.org software packages And http://github.com/petermr/opennotebook
- 12. summer school xmlsummerschool.com Slide 12 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Citizen OpenNoteBook science (plant oils) Plants emit “essential oils” (volatile chemicals) which often have antibacterial activities http://github.com/petermr/CEVOpen NOTE: all items are identified by a Wikidata item (Q number)
- 13. summer school xmlsummerschool.com Slide 13 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Search EuropePMC with getpapers for plant oils Download 200 papers (30 sec) from EuropePMC getpapers –q “((essential oil) AND (compound))” #EPMC --xml #output format –-outdir oil200/ #makes directory (CProject) –k 200 #cutoff http://github.com/ContentMine/getpapers
- 14. summer school xmlsummerschool.com Slide 14 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Download EuropePMC XML into CProject and CTrees Oil186 CProject ├── PMC4391421 CTree │ ├── eupmc_result.json download metadata │ ├── fulltext.xml full JATS text of article ├── PMC5080681 CTree │ ├── eupmc_result.json download metadata │ ├── fulltext.xml full JATS text of article ├── PMC5132230 CTree │ ├── eupmc_result.json download metadata │ ├── fulltext.xml full JATS text of article ├── PMC5203915 CTree │ ├── eupmc_result.json download metadata │ ├── fulltext.xml full JATS text of article NOTE: Tree automatically created by `getpapers` Data can be translated automatically by jats2html.xsl Ctrees are extensible. Let’s create HTML...
- 15. summer school xmlsummerschool.com Slide 15 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Citizen OpenNoteBook science (JATS XML) AMI is ContentMine’s Open XML toolkit Converts FROM XML and TO XML where possible Transform JATS-XML to HTML (normally integrated) ami-transform –p oil200 --transform nlm2html.xsl #stylesheet --input fulltext.xml --output scholarly.html http://github.com/petermr/ami3 NOTE: JATS used to be called NLM. In our demo of ami-search this is all hidden.
- 16. summer school xmlsummerschool.com Slide 16 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License PART of ContentMine stylesheet for JATS2HTML <xsl:template match=“jats:article"> <html:head> <html:style type="text/css” > a {background : #ffffff;} article {border-style : dotted;} </html:style> </html:head> <html:body> <!-- PULL for main sections --> <xsl:apply-templates select=”jats:front"/> <xsl:apply-templates select=”jats:body"/> <xsl:apply-templates select=”jats:back"/> <!-- PUSH for unknown elements --> <xsl:for-each select="*[not( self::jats:front | self::jats:body | self::jats:back)]"> <xsl:call-template name=“unknownElement”/> </xsl:for-each> </html:body> </xsl:template> NOTE: JATS has ca 250 elements, not always used wisely
- 17. summer school xmlsummerschool.com Slide 17 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License ScholarlyHTML showing JATS markup <table> <caption> <th> <tr> <xref> <p> <div> <ref>
- 18. summer school xmlsummerschool.com Slide 18 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Result of applying NLM2HTML Oil186 CProject ├── PMC4391421 CTree │ ├── eupmc_result.json download metadata │ ├── fulltext.xml full JATS text of article | ├── scholarly.html created by jats2html.xsl ├── PMC5080681 CTree │ ├── eupmc_result.json download metadata │ ├── fulltext.xml full JATS text of article | ├── scholarly.html created by jats2html.xsl ├── PMC5132230 CTree │ ├── eupmc_result.json download metadata │ ├── fulltext.xml full JATS text of article | ├── scholarly.html created by jats2html.xsl NOTE: lazy creation (a la “make”)
- 19. summer school xmlsummerschool.com Slide 19 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License ContentMine Dictionaries for searching <dictionary title="myterpenes"> <desc>terpenes searched from Wikipedia</desc> <entry id=”C127" term="(−)-menthol" wikipedia="/wiki/menthol” wikidata="Q407418” /> <entry id="C2198" term="thymol" wikipedia="/wiki/thymol" wikidata="Q408883” /> </dictionary>
- 20. summer school xmlsummerschool.com Slide 20 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Search scholarly.html for dictionary words Search for compounds, species… ami-search –p oil1000 #CProject with 1000 papers --dictionary #list of dictionaries species #species by regex country #200 countries funders #15000 funders disease #5000 diseases mydictionary/compounds.xml #2100 compounds mydictionary/plants.xml #1800 plants
- 21. summer school xmlsummerschool.com Slide 21 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License ContentMine CProject and CTrees Oil186 CProject ├── PMC4391421 CTree │ ├── eupmc_result.json download metadata │ ├── fulltext.xml full JATS text of article │ ├── scholarly.html transformed text (to be searched) │ ├── results search tree │ │ ├── search dictionary search │ │ │ ├── compound compound dictionary │ │ │ │ └── results.xml results of search │ │ │ ├── country country dictionary │ │ │ │ └── results.xml results of search │ │ │ ├── disease country dictionary │ │ │ │ └── empty.xml no results of search ... │ │ ├── species hardcoded pseudo-search │ │ │ └── binomial species (Rattus norvegicus) │ │ │ └── results.xml results of search │ │ └── word hardcoded word searches │ │ └── frequencies word cloud │ │ ├── results.xml results of search │ │ └── results.html display of search
- 22. summer school xmlsummerschool.com Slide 22 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License ContentMine XML Search results <snippetsTree> <!-- CProject --> <snippets file="oil186/PMC5080681/results/species/binomial/results.xml" > <!-- Ctree --> <result pre="ify the chemical composition, and to assess anthelmintic, antimicrobial and antioxidant effects of " exact="Thymus bovei” xpath="/html[1]/body[1]/div[2]/div[5]/p[1]" <!-- locates snippet --> match="Thymus bovei" post=" essential oil. " name="binomial"/> … NOTE: Each snippet with • semantic origin “species/binomial/” links to dictionary • separate XML result element • W3C annotations (pre, exact, post) • XPath locates it in document (W3C Annotations Rec.)
- 23. summer school xmlsummerschool.com Slide 23 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Word frequencies
- 24. summer school xmlsummerschool.com Slide 24 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Ami-search100ARTICLES 8 DICTIONARIES
- 25. summer school xmlsummerschool.com Slide 25 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Ami-search
- 26. summer school xmlsummerschool.com Slide 26 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License WIKIPEDIA
- 27. summer school xmlsummerschool.com Slide 27 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License WIKIDATA
- 28. summer school xmlsummerschool.com Slide 28 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Cooccurrence in 186 documents: Plant oils and antibacterial activity
- 29. summer school xmlsummerschool.com Slide 29 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License CONSEQUENCES: Good and Bad Universally available ! … … or not
- 30. summer school xmlsummerschool.com Slide 30 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License All the world’s 5 million FAIR Open Scientific articles (* 0.1 MB = 0.5 TB), indexed by ContentMine . Disk 30 GBP Raspberry Pi3. 50 GBP CC BY, PeterMR Disk Raspberry PI Power
- 31. summer school xmlsummerschool.com Slide 31 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License cc by-nc-sa license LabHack and Alliance Earth 1 APC = 1900 USD 1 bioreactor = 25 USD 1 Raspberry PI 55 USD 1 submission to bioRxiv Free (10 USD hidden) “a PCR machine in the UK is around £6000 but in Zimbabwe about $33000 - try convincing someone to pay APCs when they have to try CITIZENS! Zimbabwe. LabHack team from Harare Institute of Technology.
- 32. summer school xmlsummerschool.com Slide 32 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License @Senficon (Julia Reda) :Text & Data mining in times of #copyright maximalism: "Elsevier stopped me doing my research" http://onsnetwork.org/chartgerink/2015/11/16/elsevier-stopped-me-doing-my-research/ … #opencon #TDM Elsevier stopped me doing my research Chris Hartgerink
- 33. summer school xmlsummerschool.com Slide 33 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Other XML possibilities Chemistry! And “Turning PDFs into XML … is like turning hamburgers back into a cow” anon => Michael Kay (== > PeterMR)
- 34. summer school xmlsummerschool.com Slide 34 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Chem4Word creates CML (XML for chemistry) Text linked directly to chemistry
- 35. summer school xmlsummerschool.com Slide 35 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Chemistry Add-In for Word (Chem4W Chemical structures in Word documents
- 36. summer school xmlsummerschool.com Slide 36 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License So why is Chem4Word different? Open Source! Easy to use ‘Open Chemistry’ CML-based Mineable! Semantically-rich!
- 37. summer school xmlsummerschool.com Slide 37 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Worldwide Usage 400K+ downloads Colours show Word Versions in use
- 38. summer school xmlsummerschool.com Slide 38 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License New ACME Editor Advanced CML-based Molecule Editor Drop-in component Can be used in any WPF or Winforms app. Apache 2.0 License Will soon support reactions!
- 39. summer school xmlsummerschool.com Slide 39 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License Who makes Chem4Word? Now a small team of volunteers Clyde Davies Mike Williams Andy Wright Joe Townsend https://www.chem4word.co.uk
- 40. summer school xmlsummerschool.com Slide 40 Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License BUT only 10% of scholarly articles are XML The rest are HTML (usually of the worst kind) Or PDF And the images are PNGs and JPGs You can’t do anything with that??? Oh yes we can… We can turn Hamburgers into Cows! But for that you’ll have to come to PMR’s presentation