An Introduction to Artificial Intelligence for the Everyday Radiologist

An Introduction to Artificial
Intelligence for the
Everyday Radiologist
Brian Wells, MD, MS, MPH

Goals for this talk
1. The attendee will understand the basic
concepts of artificial intelligence
2. The attendee will understand basic techniques
used in artificial intelligence in radiology
imaging
3. The attendee will understand the benefits,
pitfalls, and current difficulties associated with
artificial intelligence systems related to
imaging

Why Talk About AI?
• Subjective: I think it’s the most exciting
development and emerging field in radiology.
• Objective: Companies are putting billions of
dollars into AI technologies, some of which
will directly impact healthcare. Practices will
change, and practitioners will have to adapt.
• Subjective/Objective: Radiologists that
understand will AI will likely have a
competitive advantage over those that do
not.

• “We'll see our jobs changing... If you look 10 or
25 years from now at what a radiologist is
doing, it'll probably be dramatically different.”
- Dr. Keith Dryer, vice chairman of radiology at
Massachusetts General Hospital, Boston and
Associate Professor of Radiology, Harvard
Medical School
• "Deep-learning algorithms could begin
producing radiology reports for basic studies
like mammography and chest x-rays in as soon
as five years, and for most types of imaging
studies over the next 20 years.“
- Dr. Bradley Erickson, PhD, of the Mayo Clinic,
Rochester, MN

AI and Machine Learning in Radiology. https://on-demand.gputechconf.com/gtc/2019/video/_/S9784/

AI and Machine Learning in Radiology. https://on-
demand.gputechconf.com/gtc/2019/video/_/S9784/

Not just medicine and radiology…

William Röntgen
and Anna Bertha Röntgen’s hand
(1895)
IBM’s AI-driven ultrasound analysis
software

What do we at UF Health
Jacksonville think about AI?

What Do We Think?
• ACR 2018 Project
• Resident and Attending Perceptions on Artificial Intelligence
in Radiology
• Results from the precourse survey showed 21
respondents out of 45 recipients.
• Most respondents were residents (57%), were
somewhat familiar with AI (67%), would be willing to
use it (48%), and strongly wanted to know more
(43%).
• Eighteen respondents completed the postcourse
survey (86%). Of those, 88% found the course free of
bias (6% no opinion, 6% disagree) and educational
(100%).
• After the course, 94% felt familiar with AI and wanted
to know more. 61% would be willing to use AI.

“Other” Responses
• Precourse:
• Adding time to reading studies rather than
increasing efficiency
• It is coming. It will change radiology as we know it.
Challenge is to figure out how as imagers we can
add value to the process.
• Postcourse:
• Decreased efficiency
• It will be an asset as long as radiologists develop
into consultants. Note how the AI as shown in
Watson video, gives a differential diagnosis and
does not simply repeat observations. How will
radiologists add more???

Why Learn about Artificial
Intelligence?
• Artificial intelligence (AI) has captured the
imagination and attention of doctors over the
past few years as several companies and large
research hospitals work to perfect systems for
clinical use.
• One third of healthcare AI startups raising
venture capital post January 2015 have been
working on imaging and diagnostics

The business market for medical
imaging is exploding
https://www.cbinsights.com/research/artificial-intelligence-healthcare-investment-heatmap/

The Market for AI
Companies Has Been
Progressive

Challenges facing clinicians and
radiologists
• One of the biggest problems facing physicians and
clinicians in general is the overload of too much
patient information to sift through.
• IBM researchers estimate that medical images
currently account for at least 90 percent of all medical
data, making it the largest data source in the
healthcare industry.

Why learn about artificial
intelligence?
• The scope of medical knowledge is immense

Discrepancy Rates
• Abujudeh and associates from the Department of Radiology at
Massachusetts General Hospital and Harvard Medical School
investigated discrepancy rates for the interpretation of abdominal
and pelvic CT examinations among experienced radiologists.
• Ninety examinations, which were interpreted between May 2006 and
April 2007 by one of three designated, Body fellowship-trained
expert radiologists with a mean subspecialty radiology experience of
5.7 years, were selected for review.
• The same radiologists were blinded to the previous interpretations
and were asked to reinterpret 60 examinations - 30 of their own
previously interpreted cases and 30 interpreted by their colleagues.
• The interobserver (between two different radiologists) major
disagreement rate was 26%; while, the intraobserver (disagreeing
with one’s self) major disagreement rate was 32%.
• Similar findings were showing in a University of Texas study.
Abujudeh, HH, Boland, GW, Kaewalai, R, et al. Abdominal and Pelvic Computed Tomography (CT)
Interpretation: discrepancy rates among experienced radiologists. Eur Radiol.2010;20(8): 1952-7.

Discrepancy Rates
• Platts-Mills and associates reported in The
Journal of Emergency Medicine in 2008 a 7%
major discrepancy rate for interpretation of
abdominal and pelvic CT examinations,
interpreted for patients seen through the
emergency room.
• The discrepancy rate between general
radiologists and subspecialty radiologists can
be as high as 7.7%.
Platts-Mills TF, Hendy GW, Ferguson B (2008). Teleradiology interpretations of emergency department
computed tomography scans. J Emerg Med. 2010;38(2):188-195.

Why Artificial Intelligence?
• Radiologists want a bigger role in healthcare, one
that allows them a say in patient management,
ideally one that goes from diagnosis to therapy
follow-up.
• They will get it only if they can demonstrate their
involvement adds clinical value.
• Improving patient outcomes is one route to this
goal, and artificial intelligence (AI) may be the
vehicle.

When Radiologists Were the
“Doctor’s Doctor”
• Film-based interpretation was inefficient;
however:
• We were MUCH more collaborative and true
clinical colleagues; we had a much more
complete understanding of the patient’s
clinical context
• The radiology report was secondary to person-
to-person interaction
• We provided more “precise”, patient specific
and “valued” radiology services

Has Modern Radiology “Lost Its
Way”
• Digital based radiology is certainly more efficient;
however:
• We are increasingly “isolated” from our clinical
colleagues
• We frequently have incomplete understanding of
the patient’s clinical context
• We are not truly collaborative; we are overly
dependent on the radiology report
• Our interpretations are frequently “imprecise”
and not “impactful”
• We are increasingly mired in “busy work” and
“burn out” is a real risk

Convergence of Radiology and
Pathology
• Growing connection between pathology and
radiology
• According to researchers at the University of
Pennsylvania and the Scripps Research Institute,
the two specialties should be combined into one
role called the “information specialist.”
• This individual would interpret diagnostic images
and oversee artificial intelligence disease-
screening technology.
• “If pigeons are capable of detecting
roentgenographic patterns, then radiologists
might be able to pivot their role in the diagnostic
imaging arena”

Pigeons (Columba livia) as Trainable
Observers of Pathology and Radiology
Breast Cancer Images
http://journals.plos.org/plosone/article?id=10.1371/jo
urnal.pone.0141357
• By the end of training, pigeons averaged 85-
90% accuracy.

Variability in Interpretive Performance at
Screening Mammography and Radiologists’
Characteristics Associated with Accuracy
• Fellowship-trained individuals had a
sensitivity of 88% and a false-positive rate of
11%, but non–fellowship-trained radiologists
had a sensitivity of 83% and a false-positive
rate of 9%
• http://pubs.rsna.org/doi/full/10.1148/radiol.
2533082308

Machine Learning
• Takes many forms:
• Decision trees
• Association rule
• Artificial neural networks
• Deep Learning*
• Inductive Logic
• Bayesian networks
• Reinforcement learning
• Learning classifiers
• Others

General Types of Learning
• In supervised learning, the "trainer" will present
the computer with certain rules that connect an
input (an object's feature, like "smooth," for
example) with an output (the object itself, like a
marble).
• In unsupervised learning, the computer is given
inputs and is left alone to discover patterns.
• In reinforcement learning, a computer system
receives input continuously (in the case of a
driverless car receiving input about the road, for
example) and constantly is improving.

AI does not have to be
complicated…
• Simple neural network in just 9 lines of code

• Powerful, unforeseen sources of computing power
are now available.

Transforming gaming to
Deep Learning
Dr. Andrew Ng founded and led the “Google Brain” project which developed
massive-scale deep learning algorithms. This resulted in the famous “Google cat”
result, in which a massive neural network with 1 billion parameters learned from
unlabeled YouTube videos to detect cats.

Concepts of probabilistic-based determination

Relational SQL Linkages with Probabilistic Logic

IBM Watson
• https://www.itnonline.com/videos/examples-
artificial-intelligence-medical-imaging-
diagnostics

2017 Marco Ramoni Distinguished Paper Award for Translational Bioinformatics at the AMIA
Joint Summits Meeting for the paper “ Towards Generation, Management, and Exploration of
Combined Radiomics and Pathomics Datasets for Cancer Research": Joel Saltz, Jonas Almeida, Yi
Gao, Ashish Sharma, Erich Bremer, Tammy DiPrima, Tahsin Kurc, Mary Saltz, and Jayashree
Kalpathy–Cramer.

Segmentation
• In semantic segmentation, we want to
determine the class (type of object) of each
pixel in an image.

Segmentation
• Deep neural networks are able to perform very
well on this kind of segmentation.
• Architectures often involve multiple
convolutional layers and pooling layers
• These layers compress the image into a small
neural representation of the image.
• This representation is then fed through a series of
upsampling or deconvolutional layers until we
end up with an image that is the same size as the
original image.
• The final image has multiple channels, one for
each type of object that we can classify.
• Each channel specifies whether the object
corresponding to the channel is present at each
location in the image.

Challenges to Segmentation
• Memory-intensive 3D data
• Smoothing voxel-wise predictions
Most brain segmentation models work with small regions at a time and the
prediction for each pixel is made independently of the predictions for nearby
pixels. This kind of model doesn't take into account the relation between
nearby pixels, for instance an individual healthy pixel in the middle of a
tumor is very unlikely. We can use post-processing methods to smooth the
output of the model.
• Missing data, for example a missing MRI
sequence

Challenges to Implementation
• Technical challenges also need to be addressed
• Figuring out how to establish the best source of
truth for validating results
• Determining if processing speeds will be fast
enough to be relevant for clinical practice
• Investigating whether protocol-tolerant AI
programs can be developed, and exploring
whether criteria can be established for
determining if an AI program is valid for a given
patient population.

Heatmapping
• A heatmap is a graphical representation of data
that uses a system of color-coding to represent
different values.
• Heatmaps are used in various forms of
analytics, such as to show user behavior on
specific webpages or webpage templates.
• Probabilistic maps based on designer-selected
characteristics

CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays
with Deep Learning

CheXNet
• ~2 billion procedures per year, chest X-rays are
the most common imaging examination tool
used in practice, critical for screening, diagnosis,
and management of diseases including
pneumonia.
• However, an estimated two thirds of the global
population lacks access to radiology diagnostics.

CheXNet
• CheXNet is a 121-layer convolutional neural
network that inputs a chest X-ray image and
outputs the probability of pneumonia along
with a heatmap localizing the areas of the
image most indicative of pneumonia.

CheXNet
• CheXNet is a 121-layer convolutional neural
network that inputs a chest X-ray image and
outputs the probability of pneumonia along with a
heatmap localizing the areas of the image most
indicative of pneumonia.
• CheXNet was trained on the NIH ChestX-ray14
dataset, which contains 112,120 frontal-view X-ray
images of 30,805 unique patients, annotated with
up to 14 different thoracic pathology labels using
NLP methods on radiology reports
• Original paper:
https://arxiv.org/pdf/1711.05225.pdf

CheXNet
• Collected a test set of 420 frontal chest X-rays.
Annotations were obtained independently from
four practicing radiologists at Stanford
University, who were asked to label all 14
pathologies listed by NIH.
• Performance of an individual radiologist was
evaluated by using the majority vote of the
other 3 radiologists as ground truth.
• CheXNet was evaluated using the majority vote
of 3 of 4 radiologists, repeated four times to
cover all groups of 3.

And Software is in Development for the
Interpretation of Chest CT
Source: RadLogics automated interpretation of chest CT

MGH Breast Imaging
• Can unnecessary surgeries be eliminated while still
maintaining the important role of mammography in cancer
detection? Researchers at MIT’s Computer Science and
Artificial Intelligence Laboratory (CSAIL), Massachusetts
General Hospital, and Harvard Medical School believe that
the answer is to turn to artificial intelligence (AI).
• Every year thousands of women go through painful,
expensive, scar-inducing surgeries that weren’t even
necessary.
• “The model correctly diagnosed 97 percent of the breast
cancers as malignant and reduced the number of benign
surgeries by more than 30 percent compared to existing
approaches.”
• MGH radiologists may begin incorporating the model into
their clinical practice over the next year.

“Success for artificial intelligence
(AI) in radiology will be
determined by its ability to
increase diagnostic certainty,
speed turnaround, yield better
patient outcomes, and improve
the work life of radiologists”
Journal of the American College of Radiology, February 4

The Return of the Doctor’s Doctor
• Precision medicine will make great demands on radiologists in the near
future
• Our existing human machine IT models are inadequate to address the
new requirements of precision medicine
• Do not be “seduced” by “capabilities” or “tools” (such as AI) alone.
Instead, have a “cybernetic” perspective: concentrate on solving real
world problems and achieving desired goals
• In order to fully leverage AI, existing workflows will need to be re-
engineered
• The goal: Data driven optimization human-machine cybernetic workflow
• The human knowledge worker must be “in the loop” and stay engaged in
order to optimize value to our patients

References
• https://www.auntminnie.com/index.aspx?sec=log&URL=http%3a%2f%
2fwww.auntminnie.com%2findex.aspx%3fsec%3dsup%26sub%3dpac%2
6pag%3ddis%26ItemID%3d114910
• https://www.techrepublic.com/article/why-ai-is-about-to-make-some-
of-the-highest-paid-doctors-obsolete/
• http://www.modernhealthcare.com/article/20170708/TRANSFORMATI
ON03/170709944
• https://www.the-
scientist.com/?articles.view/articleNo/30693/title/The-First-X-ray--
1895/#.WnnEyyO6W58.email
• https://www.slideshare.net/eransch/information-technology-and-
radiology-challenges-and-future-perspectives
• https://medium.com/technology-invention-and-more/how-to-build-a-
simple-neural-network-in-9-lines-of-python-code-cc8f23647ca1

References
• https://www.auntminnie.com/index.aspx?sec=sup&sub=aic&pag=dis&ItemID=119786
• https://www.auntminnie.com/index.aspx?sec=sup&sub=adv&pag=dis&ItemID=11976
2
• https://arxiv.org/abs/1711.05225
• https://www.auntminnie.com/index.aspx?sec=sup&sub=aic&pag=dis&itemId=119166
• https://www.auntminnie.com/index.aspx?sec=sup&sub=aic&pag=dis&itemId=119260
• http://news.mit.edu/2017/artificial-intelligence-early-breast-cancer-detection-1017

An Introduction to Artificial Intelligence for the Everyday Radiologist

More Related Content

What's hot (20)

Similar to An Introduction to Artificial Intelligence for the Everyday Radiologist (20)

More from Brian Wells, MD, MS, MPH (20)

Recently uploaded (20)

An Introduction to Artificial Intelligence for the Everyday Radiologist