Developing a Replicable Methodology for Automated Identification of Emerging Technologies in Healthcare
1 like624 views
The document outlines a methodology developed by an emerging technologies team to automate the identification of emerging technologies in healthcare, aimed at aiding medical librarians in accessing relevant research. It describes the processes of journal selection, term selection, and the creation of search strategies that will be used for text-mining to reveal significant concepts. Future recommendations include collaboration between medical librarians and professionals from engineering and computer science to enhance the research process.
![TEMPLATE DESIGN © 2008
www.PosterPresentations.com
Developing a Replicable Methodology for Automated Identification of
Emerging Technologies in Healthcare
P. F. Anderson1
<pfa@umich.edu>; Skye Bickett2
; Joanne Doucette3
; Pamela Herring4
; Andrea Kepsel5
; Tierney Lyons6
; Scott McLachlan7
; Carol Shannon1
; Lin Wu8
1) University of Michigan-Ann Arbor; 2) Georgia Campus-Philadelphia College of Osteopathic Medicine; 3) MCPHS University, Boston; 4) University of Central Florida College of Medicine;
5) Michigan State University; 6) Cerebros Medical Systems, Jessup, PA; 7) Ruskin College, Oxford; 8) Texas A&M University
Objectives
The Emerging Technologies Team, part of the MLA systematic review projects, was
seeking a replicable and authoritative methodology to mine PubMed for emerging
technologies relevant to health and medical librarians, and to create a methodology
to empower librarians to perform this task more easily in the future. Ideally, the
findings should reflect the range and diversity of health and medical librarians, their
workplaces, resources, and supported communities.
Organizations working in this space, such as the Gartner Group and the New Media
Consortium (Horizon Report), either have no formal methodology or do not publicly
share their strategy. Thus, it was necessary to create and document a methodology
for this purpose that can be used by MLA now and in the future.
Methods
Journal Selection Process (Flow Chart)
Emerging technologies are constantly evolving. Identifying relevant journals focused
on healthcare technologies will assist in the discovery of novel or previously
undiscovered technology terms through the text-mining process.
Term Selection Process: Emerging
Beginning with the emerging technologies search filter developed by Varela-Lema et
al, the team explored truncation, term mappings, and related terms. Alternate search
strategies were developed by four volunteers from the team. The complete team
compared, tested sensitivity and specificity of individual terms as well as [TI] vs
[TIAB] searching to achieve consensus on search terms and processes.
Using the MeSH browser, relevant structured vocabulary terms were identified as
possibilities to include in the search strategy. These terms were assigned by primary
content focus to the clusters of “emerging” or “technologies” or both, with attention
given to terms included by explosion of a higher level term.
Next Steps & Recommendations
The master search strategy has been designed in three versions (broad, narrow, and
standard), which will be tested with filters on specific topics such as the human body,
public health, education / communication, and libraries / publication. The search results
will be reduced to a text dataset comprised of titles and abstracts, to which text mining
will be applied to identify conceptual “hot spots.”
For future iterations of this project, we recommend developing teams of medical
librarians partnered with engineers, computer scientists, or related fields, and
searching the engineering literature in addition to the medical literature.
Term Selection Process: Journals
?????????????
Current Search Strategy
Sources / Resources
Find us
((Bioprospecting[MeSH] OR Drug Discovery[MeSH] OR Inventions[MeSH] OR "Patents as Topic"[Mesh] OR
“Printing, Three-Dimensional”[MeSH]) OR (“drug prospecting” OR “drug design” OR “drug repositioning” OR
“drug repurposing” OR “patent review” OR trademark OR (3d OR 3-d OR “three-dimensional” OR 3-dimensional)
print:))) OR ((1st[TI] OR advanc*[TI] OR develop*[TI] OR earliest[TI] OR early[TI] OR emergent*[TI] OR emerging
[TI] OR experiment*[TI] OR feasibilit*[TI] OR feasible[TI] OR first*[TI] OR improve*[TI] OR initial*[TI] OR initiat*[TI]
OR innovat*[TI] OR new[TI] OR newer[TI] OR newest[TI] OR newly[TI] OR novel*[TI] OR pilot[TI] OR pilots[TI]
OR preliminar*[TI] OR prototyp*[TI] OR recent*[TI] OR “single-center”[TI] OR “single-centre”[TI] OR trial*[TI] OR
validat*[TI]) AND (application[TI] OR applications[TI] OR approach*[TI] OR device*[TI] OR diagnostic*[TI] OR
image*[TI] OR imaging[TI] OR intervent*[TI] OR material*[TI] OR method*[TI] OR simulat*[TI] OR system[TI] OR
systems[TI] OR technique*[TI] OR technolog*[TI] OR test[TI] OR testabil*[TI] OR testable[TI] OR tested[TI] OR
testing[TI] OR tests[TI] OR tool[TI] OR tools[TI])) OR (BMJ innovations[Jour] OR Healthcom[Jour] OR "IEEE
Biomedical Circuits and Systems Conference : healthcare technology : [proceedings]. IEEE Biomedical Circuits
and Systems Conference"[Jour] OR “IEEE journal of translational engineering in health and medicine”[Jour] OR
“IEEE transactions on bio-medical engineering”[Jour] OR "IEEE transactions on biomedical circuits and
systems"[Jour] OR "IEEE transactions on medical imaging"[Jour] OR "IEEE transactions on nanobioscience"
[Jour] OR "IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE
Engineering in Medicine and Biology Society"[Jour] OR "IEEE transactions on systems, man, and cybernetics.
Systems"[Jour] OR "IEEE/ACM transactions on computational biology and bioinformatics / IEEE, ACM"[Jour] OR
International Conference on Pervasive Computing Technologies for Healthcare[Jour] OR "International
IEEE/EMBS Conference on Neural Engineering : [proceedings]. International IEEE EMBS Conference on Neural
Engineering"[Jour] OR "International innovation : disseminating science, research and technology"[Jour] OR
International Workshop on Databases and Expert Systems Applications[Jour] OR "... International Workshop on
Software Engineering in Health Care. International Workshop on Software Engineering in Health Care"[Jour] OR
“Proceedings / Annual IEEE Symposium on Computer-Based Medical Systems”[Jour] OR "Proceedings / BioVis,
IEEE Symposium on Biological Data Visualization ... IEEE Symposium on Biological Data Visualization"[Jour] OR
"Proceedings / IEEE International Symposium on Computer-Based Medical Systems. IEEE International
Symposium on Computer-Based Medical Systems"[Jour] OR "Surgical innovation"[Jour])
Original base search strategy:
Varela-Lema L, Punal-Riobóo J, Acción BC, Ruano-Ravina A, García ML. Making processes
reliable: a validated Pubmed search strategy for identifying new or emerging technologies. Int
J Technol Assess Health Care. 2012 Oct;28(4):452-9.
More information:
Agency for Healthcare Research and Quality (AHRQ). EPC Methods: An Exploration of the
Use of TextMining Software in Systematic Reviews. April 2016. AHRQ Publication No. 16-
EHC023-EF. https://www.effectivehealthcare.ahrq.gov/ehc/products/625/2214/text-mining-
report-160419.pdf
Arora SK, Porter AL, Youtie J & Shapira P. Capturing new developments in an emerging
technology: an updated search strategy for identifying nanotechnology research outputs.
Scientometrics 2013; 95(1): 351-370.
Huang Y, Schuehle J, Porter AL & Youtie J. A systematic method to create search strategies
for emerging technologies based on the Web of Science: illustrated for ‘Big Data’.
Scientometrics 2015; 105(3): 2005-2022.
Rotolo D, Rafols I, Hopkins MM & Leydesdorff L. Strategic intelligence on emerging
technologies: Scientometric overlay mapping. J Assoc Inf Sci Technol. Published online Dec
23 2015; DOI: 10.1002/asi.23631.
The creation of a search strategy involved identifying appropriate free text and MeSH
terms for three concept groups (emerging technologies, emerging, and technology),
and then testing these individually and in combination for sensitivity and specificity.
Specific journal titles with high specificity for the topic were hand selected for each
concept group. Inclusion/exclusion criteria were identified for the level of specificity
required for terms and titles. Concept groups were combined as: [“Emerging
Technologies” OR (Emerging AND Technologies)]. The search is limited to the most
recent 1-3 years and English language. The language limit is to avoid the challenges
of trying to apply text mining across multiple languages whose terms may not be
equivalent.
Term Selection Process: Technologies
Term Selection Process: MeSH
MLASR6 https://plus.google.com/u/0/communities/115832551443909297773
The Medical Library Association initiated a large systematic review project to assess the
level of evidence available to support the profession and practice of medical librarianship in
several very important questions. Team 6 has been assigned to explore this topic:
The explosion of information, expanding of technology (especially mobile technology), and
complexity of healthcare environment present medical librarians and medical libraries
opportunities and challenges. To live up with the opportunities and challenges, what kinds
of skill sets or information structure do medical librarians or medical libraries are required to
have or acquire so as to be strong partners or contributors of continuing effectiveness to the
changing environment?
Journals selected from NLM Catalog were based on searching relevant terms from
subgroups and emerging technologies base search. IEEE and ACM titles were added to
the journal list; table of contents were scanned for prior 3 issues or years for the inclusion
and exclusion criteria. Titles screened: 157. Titles excluded: 137. Titles retained: 20.
The same method was applied to technologies term selection. Here, certain terms
from the original search strategy were determined to not add value to the results, and
were excluded.
Special Circumstances
Emerging technologies
Unlike most of the other MLASR teams, we are not actually doing a systematic
review, and a systematic review methodology would be inappropriate for our topic.
The systematic review methodology is designed to look backwards in time,
compiling the best of the research on that topic. For emerging technologies, we
need to look to the future, rather than the past. This temporal context created
challenges which influenced every part of the project. It was made more difficult by
the fact that, while there are many think tanks which provide lists of significant
emerging technologies, none of them have or share a replicable methodology for
how they arrive at their conclusions, forcing us to attempt to create and document
one ourselves.
Text-mining
The search strategy has been developed not for the purpose of creating a data set
of citations to evaluate as relevant or not, but rather for the purpose of creating a
data set of title and abstract terms, a kind of concordance, to be analysed with text-
mining in an attempt to identify ‘hot spots’ or emerging concepts of significance. This
means inclusion/exclusion criteria relate not to the characteristics of the resulting
citation set, but rather to the purpose of developing a text file which is clearly on
emerging technologies and not muddied by other concepts, however interesting
those concepts might be.](https://image.slidesharecdn.com/mlasr6-mlanet16poster-48x48-160519193919/85/Developing-a-Replicable-Methodology-for-Automated-Identification-of-Emerging-Technologies-in-Healthcare-1-320.jpg)
Developing a Replicable Methodology for Automated Identification of Emerging Technologies in Healthcare
- 1. TEMPLATE DESIGN © 2008 www.PosterPresentations.com Developing a Replicable Methodology for Automated Identification of Emerging Technologies in Healthcare P. F. Anderson1 <pfa@umich.edu>; Skye Bickett2 ; Joanne Doucette3 ; Pamela Herring4 ; Andrea Kepsel5 ; Tierney Lyons6 ; Scott McLachlan7 ; Carol Shannon1 ; Lin Wu8 1) University of Michigan-Ann Arbor; 2) Georgia Campus-Philadelphia College of Osteopathic Medicine; 3) MCPHS University, Boston; 4) University of Central Florida College of Medicine; 5) Michigan State University; 6) Cerebros Medical Systems, Jessup, PA; 7) Ruskin College, Oxford; 8) Texas A&M University Objectives The Emerging Technologies Team, part of the MLA systematic review projects, was seeking a replicable and authoritative methodology to mine PubMed for emerging technologies relevant to health and medical librarians, and to create a methodology to empower librarians to perform this task more easily in the future. Ideally, the findings should reflect the range and diversity of health and medical librarians, their workplaces, resources, and supported communities. Organizations working in this space, such as the Gartner Group and the New Media Consortium (Horizon Report), either have no formal methodology or do not publicly share their strategy. Thus, it was necessary to create and document a methodology for this purpose that can be used by MLA now and in the future. Methods Journal Selection Process (Flow Chart) Emerging technologies are constantly evolving. Identifying relevant journals focused on healthcare technologies will assist in the discovery of novel or previously undiscovered technology terms through the text-mining process. Term Selection Process: Emerging Beginning with the emerging technologies search filter developed by Varela-Lema et al, the team explored truncation, term mappings, and related terms. Alternate search strategies were developed by four volunteers from the team. The complete team compared, tested sensitivity and specificity of individual terms as well as [TI] vs [TIAB] searching to achieve consensus on search terms and processes. Using the MeSH browser, relevant structured vocabulary terms were identified as possibilities to include in the search strategy. These terms were assigned by primary content focus to the clusters of “emerging” or “technologies” or both, with attention given to terms included by explosion of a higher level term. Next Steps & Recommendations The master search strategy has been designed in three versions (broad, narrow, and standard), which will be tested with filters on specific topics such as the human body, public health, education / communication, and libraries / publication. The search results will be reduced to a text dataset comprised of titles and abstracts, to which text mining will be applied to identify conceptual “hot spots.” For future iterations of this project, we recommend developing teams of medical librarians partnered with engineers, computer scientists, or related fields, and searching the engineering literature in addition to the medical literature. Term Selection Process: Journals ????????????? Current Search Strategy Sources / Resources Find us ((Bioprospecting[MeSH] OR Drug Discovery[MeSH] OR Inventions[MeSH] OR "Patents as Topic"[Mesh] OR “Printing, Three-Dimensional”[MeSH]) OR (“drug prospecting” OR “drug design” OR “drug repositioning” OR “drug repurposing” OR “patent review” OR trademark OR (3d OR 3-d OR “three-dimensional” OR 3-dimensional) print:))) OR ((1st[TI] OR advanc*[TI] OR develop*[TI] OR earliest[TI] OR early[TI] OR emergent*[TI] OR emerging [TI] OR experiment*[TI] OR feasibilit*[TI] OR feasible[TI] OR first*[TI] OR improve*[TI] OR initial*[TI] OR initiat*[TI] OR innovat*[TI] OR new[TI] OR newer[TI] OR newest[TI] OR newly[TI] OR novel*[TI] OR pilot[TI] OR pilots[TI] OR preliminar*[TI] OR prototyp*[TI] OR recent*[TI] OR “single-center”[TI] OR “single-centre”[TI] OR trial*[TI] OR validat*[TI]) AND (application[TI] OR applications[TI] OR approach*[TI] OR device*[TI] OR diagnostic*[TI] OR image*[TI] OR imaging[TI] OR intervent*[TI] OR material*[TI] OR method*[TI] OR simulat*[TI] OR system[TI] OR systems[TI] OR technique*[TI] OR technolog*[TI] OR test[TI] OR testabil*[TI] OR testable[TI] OR tested[TI] OR testing[TI] OR tests[TI] OR tool[TI] OR tools[TI])) OR (BMJ innovations[Jour] OR Healthcom[Jour] OR "IEEE Biomedical Circuits and Systems Conference : healthcare technology : [proceedings]. IEEE Biomedical Circuits and Systems Conference"[Jour] OR “IEEE journal of translational engineering in health and medicine”[Jour] OR “IEEE transactions on bio-medical engineering”[Jour] OR "IEEE transactions on biomedical circuits and systems"[Jour] OR "IEEE transactions on medical imaging"[Jour] OR "IEEE transactions on nanobioscience" [Jour] OR "IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society"[Jour] OR "IEEE transactions on systems, man, and cybernetics. Systems"[Jour] OR "IEEE/ACM transactions on computational biology and bioinformatics / IEEE, ACM"[Jour] OR International Conference on Pervasive Computing Technologies for Healthcare[Jour] OR "International IEEE/EMBS Conference on Neural Engineering : [proceedings]. International IEEE EMBS Conference on Neural Engineering"[Jour] OR "International innovation : disseminating science, research and technology"[Jour] OR International Workshop on Databases and Expert Systems Applications[Jour] OR "... International Workshop on Software Engineering in Health Care. International Workshop on Software Engineering in Health Care"[Jour] OR “Proceedings / Annual IEEE Symposium on Computer-Based Medical Systems”[Jour] OR "Proceedings / BioVis, IEEE Symposium on Biological Data Visualization ... IEEE Symposium on Biological Data Visualization"[Jour] OR "Proceedings / IEEE International Symposium on Computer-Based Medical Systems. IEEE International Symposium on Computer-Based Medical Systems"[Jour] OR "Surgical innovation"[Jour]) Original base search strategy: Varela-Lema L, Punal-Riobóo J, Acción BC, Ruano-Ravina A, García ML. Making processes reliable: a validated Pubmed search strategy for identifying new or emerging technologies. Int J Technol Assess Health Care. 2012 Oct;28(4):452-9. More information: Agency for Healthcare Research and Quality (AHRQ). EPC Methods: An Exploration of the Use of TextMining Software in Systematic Reviews. April 2016. AHRQ Publication No. 16- EHC023-EF. https://www.effectivehealthcare.ahrq.gov/ehc/products/625/2214/text-mining- report-160419.pdf Arora SK, Porter AL, Youtie J & Shapira P. Capturing new developments in an emerging technology: an updated search strategy for identifying nanotechnology research outputs. Scientometrics 2013; 95(1): 351-370. Huang Y, Schuehle J, Porter AL & Youtie J. A systematic method to create search strategies for emerging technologies based on the Web of Science: illustrated for ‘Big Data’. Scientometrics 2015; 105(3): 2005-2022. Rotolo D, Rafols I, Hopkins MM & Leydesdorff L. Strategic intelligence on emerging technologies: Scientometric overlay mapping. J Assoc Inf Sci Technol. Published online Dec 23 2015; DOI: 10.1002/asi.23631. The creation of a search strategy involved identifying appropriate free text and MeSH terms for three concept groups (emerging technologies, emerging, and technology), and then testing these individually and in combination for sensitivity and specificity. Specific journal titles with high specificity for the topic were hand selected for each concept group. Inclusion/exclusion criteria were identified for the level of specificity required for terms and titles. Concept groups were combined as: [“Emerging Technologies” OR (Emerging AND Technologies)]. The search is limited to the most recent 1-3 years and English language. The language limit is to avoid the challenges of trying to apply text mining across multiple languages whose terms may not be equivalent. Term Selection Process: Technologies Term Selection Process: MeSH MLASR6 https://plus.google.com/u/0/communities/115832551443909297773 The Medical Library Association initiated a large systematic review project to assess the level of evidence available to support the profession and practice of medical librarianship in several very important questions. Team 6 has been assigned to explore this topic: The explosion of information, expanding of technology (especially mobile technology), and complexity of healthcare environment present medical librarians and medical libraries opportunities and challenges. To live up with the opportunities and challenges, what kinds of skill sets or information structure do medical librarians or medical libraries are required to have or acquire so as to be strong partners or contributors of continuing effectiveness to the changing environment? Journals selected from NLM Catalog were based on searching relevant terms from subgroups and emerging technologies base search. IEEE and ACM titles were added to the journal list; table of contents were scanned for prior 3 issues or years for the inclusion and exclusion criteria. Titles screened: 157. Titles excluded: 137. Titles retained: 20. The same method was applied to technologies term selection. Here, certain terms from the original search strategy were determined to not add value to the results, and were excluded. Special Circumstances Emerging technologies Unlike most of the other MLASR teams, we are not actually doing a systematic review, and a systematic review methodology would be inappropriate for our topic. The systematic review methodology is designed to look backwards in time, compiling the best of the research on that topic. For emerging technologies, we need to look to the future, rather than the past. This temporal context created challenges which influenced every part of the project. It was made more difficult by the fact that, while there are many think tanks which provide lists of significant emerging technologies, none of them have or share a replicable methodology for how they arrive at their conclusions, forcing us to attempt to create and document one ourselves. Text-mining The search strategy has been developed not for the purpose of creating a data set of citations to evaluate as relevant or not, but rather for the purpose of creating a data set of title and abstract terms, a kind of concordance, to be analysed with text- mining in an attempt to identify ‘hot spots’ or emerging concepts of significance. This means inclusion/exclusion criteria relate not to the characteristics of the resulting citation set, but rather to the purpose of developing a text file which is clearly on emerging technologies and not muddied by other concepts, however interesting those concepts might be.