Text Mining the MAUDE Database

Signal Detection of
Adverse Medical Device
Events in the
FDA MAUDE Database
Eric Brinsfield, MS
Presenter & Research Collaborator

David Olaleye, MSCE, PhD
Author & Primary Research Statistician

SAS Institute Inc.
Cary, NC
Company Confidential - For Internal Use Only
Copyright © 2010, SAS Institute Inc. All rights reserved.

Disclosure Statement

 Both presenters are employees of SAS Institute
 We have no conflicts of interest

Disclaimer

The views and opinions expressed in the following
PowerPoint slides are those of the individual presenters
and should not be attributed to ISPE or to SAS Institute.

2


Study Objectives
Determine if text mining can be used to:
 Detect signals of adverse events in
spontaneous reporting data
 Better understand or triage signals
generated by traditional disproportionality
methods

Phase 1:
 Evaluate unsupervised text mining
 Using FDA MAUDE database
 Focused on stents
3

3

Definitions
 Safety Signal
 “A report or reports of an event with an unknown
causal relationship to treatment that is recognized as
worthy of further exploration and continued surveillance.”
» Council for International Organizations of Medical
Sciences (CIOMS)
 “Recognition” is often the results of
 Analytical or automatic signal detection methods that look
for unexpected patterns in data sources such as:
» Spontaneously reported data
» Observational healthcare data
» Insurance claims data

4


Methods

 Data Source = MAUDE
 FDA Spontaneous Reporting System Database for Med. Devices
MAUDE - Manufacturer and User Facility Device Experience
 Contains the narrative entered by the reporter

 Target Devices
 endovascular graft system and coronary stent devices
 devices classified as a stent in the “product_category_code”:
» MAF, MIH, NIN, NIO, NIP, NIM, and NIQ
» http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPCD/classif
ication.cfm?ID=896

5

5

MAUDE (2001-2008)
Spontaneous “safety” reports on medical devices
Strengths:
 Only surveillance system which covers devices
marketed in the entire US
 Largest number of case reports on adverse outcomes
and malfunctions for medical devices
 Provides opportunity to detect signals of new, rare
and unusual adverse clinical outcomes
» Usually warrants further investigation
 Includes narrative description of events
» Better than AERS which does not include narrative

6


Characteristics of Spontaneous Report
 Suitable for hypotheses generation; not for confirmation or
rates computation
 Exhibit under-reporting and other reporting biases; lack of
control group, etc.
 Data quality, ascertainment, accuracy and completeness of
information are usually poor
 Includes events and incidents not causally related to
medical device exposure
 Does not distinguish between label versus off-label uses of
approved products
 Contains minimal or no patient history and other potential
causal factors
 Do not provide estimates of exposure (worse in drugs) 7


Study Design: Case-Series
Source Population
All MAUDE reports received between 2006 – 2010
(N=35954)

Study Device Cohort Study Events
MAUDE reports for stents • Death (D)
with product codes: • Injury (I)
MAF, MIH, NIN, NIO, • Malfunctions (M)
NIP, NIM, and NIQ • Other (O)

Device-Adverse Outcomes Pairs
(N=28)
8


Analysis Steps

1. Attempt analysis using text mining alone
 Ignore device
 Evaluate if general text mining provides any insights
2. Perform standard disproportionality analysis on
structured data
 PRR, EBGM, Adj. Residual
3. Identify device-AE pairs that have:
 High scores
 Especially in the “Other” classification
4. Investigate the device with text mining
 Include all outcome classifications

9


Text Mining Process

 Parse terms to create documents-terms frequency matrix
 Use singular value decomposition (SVD) to measure
association and perform hierarchical clustering
 Use entropy method to cluster SVDs for documents
classification

10


Sample Narrative - Injury
Example of Manufacturer Report - Injury
 the product labeling for a p154 states that this product is indicated for
use in pts with obstruction of major biliary ducts. the product labeling
also states that the stent may be increased post-placement by
expanding with a larger diameter balloon. the following was obtained
through conversation with the user facility on 1/15/98. after deciding that
a ptca procedure in a renal artery did not yield adequate results, the md
attempted to place a medium biliary stent in the artery. the physician
reported to co. that he had difficulty visualizing the stent and that it was
difficult to place. he deployed the stent, but was not satisfied with the
outcome. in response, he decided to place a second stent inside of the
first. however, the stents interlocked and the physician decided to
remove both stents, he was able to withdraw the stent up to sheath tip in
the femoral artery, but needed a vascular surgeon to completely remove
them. info regarding the type of removal procedure has not been
provided to co. the physician further stated that he believes that the first
stent had not fully opened. it was mounted on a meditech glidex balloon.

11


Concept Linking and Exploration

12


Text mining clusters over all reports

13


Text Mining Over All Reports

 Results are interesting
 But inconclusive
 Lose track of the device
 But may detect new trends for all stent devices in study

Cannot really do comparisons due to lack of denominator.
(Same problems as always with spontaneous reports.)

Next, run disproportionality to narrow the focus…

14


Disproportionality Results for
Carotid Stent (NIM)

Adjusted MGPS
Adverse Event Freq Residual (EBGM)
Death 276 0.8 0.86728

Injury 2043 1.1 1.130264

Malfuntion 271 0.4 0.424803

Other 24 1.6 3.648191

* Adjusted Residual: Flagged at values over 1.5
EBGM: Flagged at values over 2.0

15


Target for Text Mining Evaluation

 NIM showed high proportion of “Other”
 Based on relative percentage of reports
 Based on signal scoring algorithms
 All methods suggested a flag
 Although only 24 cases, the method could show promise
 Run text mining against all NIM reports
 Include all outcomes to fully understand reports
 Look for possible explanations or hypotheses

16


N
Text Mining Clusters for NIM
+percutaneous +unk dissection performance +unstable repaired 376
+malfunction 'device remains implanted'
reactions collapsing +'premature deployment' cracks +'product quality 41
issue' replaced performance collapsed
malfunction fractures +fracture +'inaccurate delivery' fractured 846
malfunctions drift +'premature deployment'
+normal dissection +unk 'no information' +na collapsing unk fractured 69
'device issue' broke 'no known device problem' +break broken +'shaft 227
break' reaction reactions
abnormal +fracture +continuous +bent cracks unk +break fractured 65
filter +na 'difficult to advance' breaks reaction +bent +malfunction 150
replaced
'device remains implanted' collapsed 'no flow' performance +crack 44
+break +collapse filter
fractured repair +crack breaks +collapse +'product quality issue' 'no 31
flow' reaction 17


18


Manual Review of Text for “Other”

 Most were not adverse events that persisted
 Some seemed like “FYI” reports.
 Two included notification of a formal study
 Most patients still had the stent in place (assumed)
 Some cases of installation problems:
 Potential installer error
 Most did not involve an adverse event

19


Conclusion

 Text mining shows promise for recognizing
primary words and patterns
 Hard to form hypotheses from bulk text mining on
spontaneous database
 Combination with disproportionality analysis
creates signals that can be further analyzed with
text mining
 Terms in the “Other” category overlap with other
categories

20

20

Next Steps

Need further analysis that includes:
 Large target group for further triage
 “Other” was too small in this case
 Preferred term matching and encoding
 To clean up fuzziness and reduce clusters
 Content categorization
 Look for more structure and combine with ontologies
 Sentiment analysis
 Determine if overall sentiment was good or bad

21

21

Final Thoughts
“What the future portends is more and more
information — Everests of it. There won’t be
anything we won’t know. But there will be no
one thinking about it.”
From:
New York Times - August 13, 2011
The Elusive Big Idea
By NEAL GABLER

Neal Gabler is a senior fellow at the Annenberg Norman Lear Center at the University of
Southern California and the author of “Walt Disney: The Triumph of the American Imagination.”

We need to help make time for thinking.
22


Thank You
Contacts:

Eric Brinsfield  eric.brinsfield@sas.com

David Olaleye  david.olaleye@sas.com


Text Mining the MAUDE Database

More Related Content

Viewers also liked (13)

Similar to Text Mining the MAUDE Database (20)

Text Mining the MAUDE Database