ChemExtractor: Enhanced Rule-Based Capture and Identification of PDF Based Property Data
Download as PPTX, PDF1 like229 views
The document discusses the ChemExtractor project, which enhances the extraction and identification of chemical property data from PDF files using rule-based methods and regular expressions. It highlights the motivation behind the project, such as the need to leverage curatorial investments in chemical data, and outlines the research approach, including data analysis and the contextualization of captured data with metadata. The project aims to facilitate higher accuracy in capturing significant datasets and is intended for community use.
![ Implemented in every programming language
Relatively uniform implementation
Uses syntax to create regex string that matches
and/or captures characters in string
Groups of characters – ., [A-Z], [a-z], [0-9]
Character classes w, d, s, h
Repetition - ? (optional), + (1 or more), * ( or more)
Capture group – ()
Regular Expressions (Regex)](https://image.slidesharecdn.com/chemextractor-170503144934/85/ChemExtractor-Enhanced-Rule-Based-Capture-and-Identification-of-PDF-Based-Property-Data-7-320.jpg)
![ We write rules for specific line formats constructed using
Rule templates – define basic structure of a line, i.e.
how many blocks of text that need to be captured
Rule snippets – define small regex strings that capture a particular
format of text
Rulesets are list of rules with associated actions indicating
sequentaily process text line by line
Rules and Rulesets
^@B1@h+@B2@h*@B3@h*@B4@h*@B5@h*@B6@$
(?:[A-Z][a-z]{0,2}d{0,3}h?)+(?:.?d*.?d*H2O)](https://image.slidesharecdn.com/chemextractor-170503144934/85/ChemExtractor-Enhanced-Rule-Based-Capture-and-Identification-of-PDF-Based-Property-Data-11-320.jpg)
ChemExtractor: Enhanced Rule-Based Capture and Identification of PDF Based Property Data
- 1. ChemExtractor: Enhanced Rule-Based Capture and Identification of PDF Based Property Data Stuart J. Chalk, Department of Chemistry University of North Florida schalk@unf.edu 253rd ACS Meeting April 2017
- 2. Outline Motivation Research Approach Analyzing Tabular Data Types of Data Regular Expressions (Regex) Rules and Rulesets Examples of Data Extraction Contextualizing Data Data Storage (MySQL) Data Representation (SciData) Conclusion Funding for this project provided by
- 3. Motivation The Landholt-Börnstein Database is +450 volumes of curated chemical property data (18?? to date) With the move to data-driven science it is imperative we leverage the time invested in, and scientific quality of, the curation of this data This high value data is locked in PDF files This data can be made more useful if it is extracted with its metadata (chemical system, original reference, physical property with unit, etc.)
- 4. Optical Character Recognition (OCR) is a standard process to extract text from scanned images Accurate extraction of text using OCR in a page layout format not only captures the text but also the inferred relationships between tabular data Utilize regular expression (regex) analysis of tabular text to capture data and its position relative to other data Contextualize captured data with metadata encoded based on layout of table and string format Research Approach
- 5. Analyzing Tabular Data Chemical Metadata Series of Property Data Condition Series Condition Property Data Reference
- 6. Properties and Units Conditions, data, supplemental data Equation coefficients and variables Chemical properties (MW, BP, MP) Annotations Table headers, column headers, data notes Chemical Metadata Formula, name, CASRN Context Metadata Table #, refcodes, property headers, component # Types of Data
- 7. Implemented in every programming language Relatively uniform implementation Uses syntax to create regex string that matches and/or captures characters in string Groups of characters – ., [A-Z], [a-z], [0-9] Character classes w, d, s, h Repetition - ? (optional), + (1 or more), * ( or more) Capture group – () Regular Expressions (Regex)
- 11. We write rules for specific line formats constructed using Rule templates – define basic structure of a line, i.e. how many blocks of text that need to be captured Rule snippets – define small regex strings that capture a particular format of text Rulesets are list of rules with associated actions indicating sequentaily process text line by line Rules and Rulesets ^@B1@h+@B2@h*@B3@h*@B4@h*@B5@h*@B6@$ (?:[A-Z][a-z]{0,2}d{0,3}h?)+(?:.?d*.?d*H2O)
- 16. Examples of Data Extraction
- 17. Examples of Data Extraction
- 18. Examples of Data Extraction
- 19. Examples of Data Extraction
- 20. This data… … is detected by rule ‘Temperature (K) & refcode’ Contextualizing the Data T/K 298.15 () () () () () () () () 81V1 ()
- 26. Higher accuracy capture of property data and equation data (350,000 property data points, 10,000 equations) Integrated chemical and reference metadata Can be applied to other PDF-based curated datasets Open website for community use this summer Implementation for research article capture of chemical property data Conclusion
- 27. schalk@unf.edu Phone: 904-620-1938 Skype: stuartchalk LinkedIn: https://www.linkedin.com/in/stuchalk ORCID: http://orcid.org/0000-0002-0703-7776 Questions?