MongoDB_Talk_ValidatingAnOpenSociety_112916_Final

© 2016. 8 Path Solutions LLC.© 2016. 8 Path Solutions LLC.
Data, Decisions, and MongoDB
Tuesday, November 29, 2016
© 2015 8 Path Solutions LLC. All Rights Reserved.

Validating
an
Open
Society

Jennifer Shin
Founder, 8 Path Solutions
Senior Principal Data Scientist, Nielsen

●  Background
š Native New Yorker
š Undergraduate degree in Economics, Mathematics & Creative
Writing from Columbia University
š Graduate degree in Statistics from Columbia University
Introduction

●  Professional Experience
š Founder & Chief Data Scientist at 8 Path Solutions
š Senior Principal Data Scientist at Nielsen
š Management consultant at Fortune 100 companies
š Top Contributor for IBM Data Magazine
š Faculty in the MIDS Graduate Program at UC Berkeley
Introduction

●  Recent Talks & Presentations
š Institute of Computational & Experimental Research in Mathematics
(ICERM) at Brown University
š TDWI Accelerate 2016
š Data Dialogs Conference – UC Berkeley
š IBM World of Watson 2016
Introduction

š  Adverse Drug Reactions (ADR)
š  FDA Adverse Events Reporting System (FAERS)
š  openFDA API
š  openFDA + MongoDB
Today’s Talk

š Monitoring the safety of medicinal products
š Adverse Drug Reactions (ADR):
unwanted, uncomfortable, or dangerous effects that a drug may have
š In the US, 3 to 7% of all hospitalizations are due to ADR1
Pharmacovigilance

DATA SOURCE
š  FDA Adverse Event Reporting System (FAERS)
●  A computerized information database designed to support the FDA's
post-marketing safety surveillance program for all approved drug &
therapeutic biologic products
●  Used to monitor for new adverse events and medication errors that
might occur with these marketed products

DATA SOURCE
Option 1: Quarterly FAERS Data Files
○  Available each quarter from the FDA
○  Data from 2004 to 2012 available in ASCII/SGML
Data after 2012 available in ASCII/XML

DATA CHALLENGES
○  Requires downloading and consolidating quarterly reports in a
databases.

DATA ISSUES
○  Duplicate Reports
○  Spelling errors
○  Inaccurate information
○  One field for all drug names (e.g. Brand Name & Generic) and
active ingredients
○  Multiple drugs included in a single report

openFDA API

š  Beta launch - June 2014
š  New website
Food & Drug Administration openFDA API

š  Facilitate access and use of big important FDA public datasets by
developer, researchers, and the public through harmonization of
data across disparate FDA datasets provided via application
programming interfaces (APIs)
API OBJECTIVES

š  Drug adverse events: Reports of drug side effects, product use
errors, product quality problems, and therapeutic failures.
š  Drug product labeling: Structured product information, including
prescribing information, for approved drug products.
š  Drug recall enforcement reports: Drug product recall enforcement
reports.
API DATA

š  Drug product labeling: Structured product information, including
prescribing information, for approved drug products.
š  Drug recall enforcement reports: Drug product recall enforcement
reports.
API DATA
š  Drug adverse events: Reports of drug side effects, product use
errors, product quality problems, and therapeutic failures

API DATA
○  Access to FAERS database using API calls

API DATA
3 ways to download data from openFDA.
š Download manually.
There’s a downloads section on each endpoint’s openFDA.
š Write code to download the data automatically.
Use a special API query (see below) to get a list of all the current
data files for each endpoint.
š Synchronize with the openFDA S3 bucket.

DATA ISSUE SOLUTION
Drug Name Harmonization Process
○  Benefits
•  Harmonizes the FAERS data on drug identifiers using other
data sources, such as NDC & RxNorm
•  Separate data fields for brand names, generic names, and
active ingredients

DATA ISSUE SOLUTION
○  Limitations
•  Cannot harmonize misspelled drug names
•  Validation process requires using FAERS data files
- Not necessarily easier

CASE STUDY: DROSPIRENONE

FAERS vs. OPENFDA
FAERS Data Files
Data From 2004 Q1 to 2012 Q3
2 out of the 7 Reports:
DEMO, DRUG, REAC, RPSR, THER, OUTC, INDI
Consolidated using SQL Server 2012

FAERS vs. OPENFDA
Drug Name Mapping
DRUGNAME

YAZ

DROSPIRENONE
W/ETHINYLESTRADIOL
(YAZ)

YAZ
/06358701/

YAZ
BAYER
HEALTHCARE

YAZ
N/A
BAYER
HEALTHCARE

YAZ
(24)

YAZ
(DROSPIRENONE
+
ETHINYLESTRADIOL
20!G
(24+4)
[YAZ]

YAZ
(DROSPIRENONE/ESTRADIOL)

YAZ
(ORAL
CONTRACEPTATIVE
NOS)

FAERS vs. OPENFDA
Data Fields
FAERS Data Files openFDA API
Brand Name
Yaz

DRUGNAME

pa.ent.drug.openfda.brand_name

Generic Name
Drospirenone
Ethinyl

Estradiol

DRUGNAME

pa.ent.drug.openfda.generic_name

Case Report
Identifier

ISR

safetyrepor.d

FAERS vs. OPENFDA
○  Inconsistent Query Data
•  Running the same query on 8/03/14 &
8/10/14 produced different results

FAERS vs. OPENFDA
○  Inconsistent Query Data
•  No information as to the cause of these
changes could be found on the FDA website
•  According to the Github records, there were
no updates made between these two dates
•  For our brand name data analysis, the most
recent results were selected

DATA ISSUES SOLUTION?
○  FAERS Data Files vs. openFDA API Query
•  Cannot harmonize misspelled drug names
•  Validation process requires using FAERS data files
- Not necessarily easier

RESULTS 1:
BRAND NAME

RESULTS 1: BRAND NAME
1
0
20
105
2,502
2,321
1,472
8,857
6,750
0
0
19
215
2,498
2,261
1,365
7,881
5,551
2004 2005 2006 2007 2008 2009 2010 2011 2012
openFDA FAERS
Comparing Reports for Yaz from Q1 2004 to Q3 2012

RESULTS 1:
BRAND NAME
openFDA API
query results for
safetyreportid
“4990905-5”

RESULTS 1:
BRAND NAME
DRUGNAME for ISR number “4990905” only includes
DROSPIRENONE AND ETHINYL ESTRADIOL
openFDA API
query results for
safetyreportid
“4990905-5”

RESULTS II:
GENERIC NAME
QUERY
openFDA

Results

Initial Query
hOps://api.fda.gov/drug/event.json?search=receivedate:[20040101+TO+20120930]+AND+

pa^ent.drug.openfda.generic_name:”DROSPIRENONE+ETHINYL
+ESTRADIOL”&count=pa^ent.drug.openfda.brand_name

Total:

714

Yaz:

107

RESULTS II:
GENERIC NAME
QUERY
openFDA

Results

Initial Query
hOps://api.fda.gov/drug/event.json?search=receivedate:[20040101+TO+20120930]+AND+

paênt.drug.openfda.generic_name:”DROSPIRENONE+ETHINYL
+ESTRADIOL”&count=paênt.drug.openfda.brand_name

Total:

714

Yaz:

107

Revised Query
hOps://api.fda.gov/drug/event.json?search=receivedate:[20040101+TO+20120930]+AND
+paênt.drug.openfda.generic_name:"DROSPIRENONE”+AND
+paênt.drug.openfda.generic_name:"ETHINYL”+AND
+paênt.drug.openfda.generic_name:"ESTRADIOL”&count=paênt.drug.openfda.brand_na
me

Total:

31,051

Yaz:

22,028

Results
The drug harmonization process incorrectly associated reports for
Drospirenone Ethinyl Estradiol with the drug Yaz.
š Raises concerns about the drug harmonization process for Yaz as well
as other drugs
š Further study is needed to validate the accuracy of the openFDA data

RESULTS II:
GENERIC NAME
The reported cases for Yaz from
the openFDA API and FAERS
Data Files varied widely when
compared based on the year of
the report.
š For 2006, the API only included
105 cases, which is 51% less
than the 215 cases in FAERS.
š For 2011, the API included
8,857 cases, which is 12.4%
more than the 7,881 cases in
FAERS.

Data, Trust, and Reproducibility

Which is better?
@8PATHSOLUTIONS
š  Traditional methods vs. newer approaches
š  Data processing & data validation
š  Access via API vs. database
š  Implications for pharmaceutical research, data science,
data technology & development

Data Dependence
š  Risk of relying on API data
EX: http://download.open.fda.gov/

openFDA Data
3 ways to download data from openFDA.
š Download manually.
There’s a downloads section on each endpoint’s openFDA.
š Write code to download the data automatically.
Use a special API query (see below) to get a list of all the current
data files for each endpoint.
š Synchronize with the openFDA S3 bucket.

MongoDB
š Collecting query records
š Storing query results
š Setting up data environment

LIVE DEMO

LIVE DEMO
openFDA API website
https://open.fda.gov/index.html
FDA’S Example Query
https://open.fda.gov/api/reference/#example-query

LIVE DEMO
FDA’S Example Query: https://open.fda.gov/api/reference/#example-query
Original Query
https://api.fda.gov/drug/event.json?
search=patient.drug.openfda.pharm_class_epc:"nonsteroidal+anti-inflammatory
+drug"&count=patient.reaction.reactionmeddrapt.exact
Our Query
https://api.fda.gov/drug/event.json?search=receivedate:[20040101+TO+20120930]+AND
+patient.drug.openfda.brand_name:"Yaz"

© 2016. 8 Path Solutions LLC.© 2016. 8 Path Solutions LLC.š https://api.fda.gov/drug/event.json?
api_key=83k0zAKbRQbk5rCbedjpDs8DdKwoagWvojeW2ATf&search
=receivedate:[20040101+TO+20120930]+AND+receiptdate:
[20040101+TO+20120930]+AND+patient.drug.medicinalproduct:
%22YAZ%22&count=patient.drug.openfda.brand_name.exact

Footnotes
š  1. http://www.merckmanuals.com/professional/clinical-pharmacology/adverse-drug-
reactions/adverse-drug-reactions

MongoDB_Talk_ValidatingAnOpenSociety_112916_Final

More Related Content

Similar to MongoDB_Talk_ValidatingAnOpenSociety_112916_Final (20)

MongoDB_Talk_ValidatingAnOpenSociety_112916_Final