Using publicly available resources to build a comprehensive knowledgebase of chemical information
1 like201 views
The document discusses the development of a comprehensive knowledgebase for chemical information using publicly available resources while addressing the variability in data quality and reliability. It outlines a systematic and automated data mining workflow utilizing Python scripts, HTML parsing, and the ChemSpider API to standardize and validate chemical data. The authors have created tools for data collection, such as a command line interface for resolving chemical names and identifiers, as well as a chemical validation and standardization platform.
Using publicly available resources to build a comprehensive knowledgebase of chemical information
- 1. Using publicly available resources to build a comprehensive knowledgebase of chemical information by B. Sattarov, R. Zakharov and V.Tkachenko Science Data Software Abstract There is a variety of public resources on the Internet which contain information about various aspects of chemical, biological and pharmaceutical domains. The quality, maturity, hosting organizations, team sizes behind these data resources vary wildly and as a consequence content cannot be always trusted and the effort of extracting information and preparing it for reuse is repeated again and again at various levels. This problem is especially serious in applications for QSAR, QSPR and QNAR modeling. On the other hand authors of this poster believe, based on their own extensive experience building various types of chemical, analytical and biological databases for decades, that the process of building such knowledgebase can be systematically described and automated tool for building a comprehensive knowledgebase of chemical information. We have developed data mining workflow to collect and standardize chemical data from open sources, using several simple python scripts which will be included in open source library. Data collection was carried out by HTML parsing and by using ChemSpider API. We also used python version of Chemical Validation and Standardization Platform developed by us to standardize collected data. Our ChemScrapper allowed us to resolve 19.85% names of biologically active compounds from MESH 2017 dataset and to save this data into json and handy sdf format. Chemical Validation and Standardization Platform (CVSP), which we used to standardize chemical structures, can also be used as stand-alone platform for SMIRKS-based standardization of any dataset, thanks to the visual implementation of its python version functionality on Jupyter. You can see every standardization rule applied as SMIRKS string simply by clicking on SMIRKS button as well as download standardized dataset as *.sdf file by checking corresponding folder. Example json—output with mol block, properties and synonyms Example CLI Example input One of the most productive data mining tools we have created works with ChemSpider web API. It allows user, looking for a chemical structures/data, to deal only with convenient com- mand line interface written in Python, in order to resolve chemicals names and identifiers or to find new data for QSAR/QSPR analysis or any other purpose that requires . API HTML Parsing CVSP Standardization Data collection Comprehensive knowledgebase of chemical information Open Science Data Repository (OSDR) Comprehensive distributed semantic knowledgebase of scientific information with built-in Machine Learning capabilities