How to select , read & review an article

How to select , read &
review an article ?
Dr.Venugopalan P P
Director & Lead consultant
Emergency Medicine
Aster DM Healthcare

Background
❏ Emergency physicians provide care for
patients with a wide variety of medical
conditions in diverse clinical scenarios
❏ The wide scope of practice and resultant
required breadth of knowledge demand
frequent use of the latest medical literature
❏ Many specific reasons exist why an
emergency physician might review the
literature on a particular topic

Why E P should know how to review an article ?
❏ To understand the pathophysiology, etiology, or
clinical course and features of a disorder
❏ To learn how experts recommend handling a
clinical problem
❏ To learn the benefits of a new diagnostic test and
how it relates to existing technology
❏ To evaluate the safety, efficacy, benefits, risks,
and cost of new diagnostic or therapeutic options

Objectives
❏ Evaluating the literature is closely related to the recent
emphasis on evidence based medicine
❏ Assist the emergency physician in analyzing research
articles
❏ Integrate them into their own practice
❏ Outlines the elements of the practice of evidence based
medicine
❏ Translate a clinical question into one addressed by
research studies
❏ Assists the reader in critically evaluating research studies
and in identifying their limitations

EBM
❏ Evidence based medicine
offers an objective way to
determine and maintain
consistently high quality and
safety standards in medical
practice
❏ Speeds up the process of
transferring clinical research
findings into practice
❏ Emergency
physicians must
know how to
approach a
clinical question
from an
evidence based
medicine
perspective

Evidence based medicine process - 5 Steps
1) Formulation of a clinical
question
2) Locating the best
evidence
3) Critically appraising that
evidence
4 )Acting on the
evidence
5 ) Critiquing the result
of the process

Question formation - 4 parts
1) What is the Problem or
Patient profile ?
2) What is the Intervention ?
3) What is the Comparison
intervention ?
4) What are the Outcomes ?
P
I
C
O

How to search existing evidences to answer a
particular question?
❏ The reader needs to decide whether to look up
articles in the original literature or to look up
secondary resources that have already searched
and synthesized primary information
❏ Reviews are written by an expert in the field subject
to a review process and updated regularly
❏ Some of them are specifically targeted to particular
specialties

Cochrane Collection
❏ Groups and fields that
are specifically relevant
to emergency medicine
Physicians' Information and
Education Resource, (PIER)
❏ A series of modules
representing common
problems encountered in
internal medicine
Rowe BH, Brown MD. A primer on the Cochrane Collaboration, its new
priorities in out of hospital and emergency health, and the role of Annals of
Emergency Medicine. Ann Emerg Med. 2007 Mar. 49(3):3514. [Medline].
Badgett RG, Mulrow CD. Welcome, PIER, a new Physicians' Information and
Education Resource. Ann Intern Med. 2002 Apr 2. 136(7):5534. [Medline].

PubMed
❏ Embedded filters[1]
❏ Filters allow readers to choose
➢ Clinical study topic
➢ Interest is in Etiology, Diagnosis,
Therapy, Prognosis, or Clinical
prediction
❏ One can even control the sensitivity
or specificity of the search[2]
1. Haynes RB, Wilczynski N, McKibbon KA, Walker CJ,
Sinclair JC. Developing optimal search strategies for
detecting clinically sound studies in MEDLINE. J Am Med
Inform Assoc. 1994 NovDec. 1(6):44758. [Medline].
2. Wong SS, Wilczynski NL, Haynes RB, Ramkissoonsingh
R,. Developing optimal search strategies for detecting
sound clinical prediction studies in MEDLINE. AMIA Annu
Symp Proc. 2003. 72832. [Medline].

Measure the quality of Evidence
1. US Agency for Healthcare
Research Quality
2. US National Health Service
❏ Simple and easy to follow
❏ Rate the quality of research
studies performed to date on
similar scales of I to IV / V
❏ Higher level classifications
require prospective randomized
controlled trials

How to search existing evidences to
answer a particular question?

❏ Categorizing
the quality of
data allows
the reader to
determine
how much
confidence to
place around
the conclusion
presented

Resources
❏ Cochrane Collaboration
❏ National Library of
Medicine – Gateway
❏ Open Clinical
❏ BestBETs

How to select an article ?
The selection of a medical
database depends
1. Type of question
2. Ease of use for a
particular problem
3. Need to limit the search to
the highest quality
studies
Physicians must
understand
❏ Basic concepts
concerning critical
analysis of the medical
literature
❏ Important elements of
Study design,
Biostatistics, and
Critical analysis of
clinical research

Primary
Survey
Secondary
Survey
❏ A very brief initial review or primary survey delivers quick answers
❏ A more detailed examination or secondary survey
❏ Reviewing the literature, these two efforts correspond to
determining whether the article is worth an in depth reading,
and if so, a thoughtful review.

Primary survey - Initial
evaluation & brief
overview - Read
❏ Title
❏ Authors
❏ Abstract

Analyze the title
❏ Is this article potentially
interesting?
❏ Possibly useful in practice?
❏ If not, reject it and move
on to the next article.

Review the list of authors
❏ One or more authors may be familiar
❏ Do the authors have a track record of
thoughtful research or teaching in
this area?
❏ If so, definitely continue
❏ If not, the article may still have value,
particularly if the journal is refereed
and has a good reputation- Continue

Read the summary or abstract beginning with
the conclusion
Then, answer the following question
1. Is the conclusion, if valid, important to
clinicians?
2. If the results are true, determining how useful
they are?
3. More specifically, is the primary outcome
measured important to you?
4. Do the interventions make sense?
5. Can the information be generalized to your
population of interest?

Brief overview
“The conclusion should
address the stated objective or
goal of the study identified in
the first sentence of the
abstract and should be
supported by the results”
An article has passed this basic filtering process, proceed to the
secondary survey, a more comprehensive review of the article.

How to read an article ?
Secondary
Survey
“More
comprehensive
Review”
1) Introduction / Background
2) Method
3) Study Sample
4) Treatment allocation
5) Outcome
6) Statistical Analysis
7) Results
8) Discussion
9) Conclusion

How to read an article ? 1. Acquaint the reader with the
problem under study
2. Explain reasons for conducting
the investigation
3. It should frame the context of
the study
4. Explain why the topic is
important
5. It should explain what is known
about the topic, and what is
unknown
6. It should identify the specific
question (research objective,
goal of the study, hypothesis)
to be evaluated.
Secondary
Survey
Introduction/
Background

Secondary
Survey :
Introduction/
Background
❏ The statement of the study
objective should clearly
identify the study sample,
the primary outcome, and
the intervention being
evaluated
❏ The methods must be
designed to answer that
question
❏ The investigator's
conclusions should not
extend beyond the stated
objective

Secondary
Survey :
Methods
Provide a glimpse into the
internal workings of the study
Most major journals clearly
indicate how they expect the
methods section to be
structured.
Minor variations exist in most request type of
study design, study sample including study
setting, treatment allocation decisions,
outcome measure(s), and statistical test
selection

Secondary
Survey :
Methods
❏ What is the research design?
❏ Is it a descriptive or comparative
study?
❏ Single or multicenter?
❏ What is the timeline relationship
between the occurrence of the
events being measured and their
assessment by the study?

❏ How were the subjects and the
controls selected?
❏ Are the entry and exclusion criteria
sufficiently clear to describe the target
population?
❏ Is the study site sufficiently similar to
the reader's practice
❏ Study's results, if valid, would apply to
patients in the reader's practice?
Secondary
Survey :
Study Sample

❏ The reviewer must be able to precisely visualize the
sample under investigation based on the authors'
description of entry and exclusion criteria
❏ Entry criteria describe the population of patients
represented and enable the reader to determine
whether the study sample sufficiently resembles
their clinical practice to allow extrapolation
❏ Exclusion criteria help to ensure that the study
sample is as homogeneous as possible, to identify
patient subsets to which study results should not be
extrapolated
❏ Ensure patient safety by excluding individuals for
whom participation would be contraindicated or
dangerous
Secondary
Survey :
Study
Sample

❏ Not all studies have control groups
❏ If a control grpoup:
What is the nature
❖ Concurrent
❖ Paired or matched to a study
subject
❖ Treated with placebo or another
active treatment?
Secondary
Survey :
Study Sample

1. How was the treatment assigned?
2. Was randomization used?
(If so, confounding variables may
still be present but are less likely to
affect the outcomes)
3. While administering the treatment,
were the subjects, researchers, or
both blinded?
Secondary
Survey :
Treatment
alloction

Secondary
Survey :
Outcome
❏ Primary or Several secondary
❏ Primary outcome may be final
❏ Simple or Composite
❏ Difine within a time frame
❏ Does the measured outcome is
valid ?
❏ Is it truly represents the
phenomena of interest ?

❏ Does it intuitively make sense?
❏ Has the outcome measure
previously gone through a process
of "validation"
❏ Does it has been used in other
published research on the same
topic?
Secondary
Survey :
Outcome

❏ The authors should clearly state how it
was measured
❏ Measurements should always be as
precise and reproducible as possible
❏ The measurements are free of bias
❏ Measures that are prone to subjective
differences bring their own set of
challenges
❏ Authors should standardize
measurements
❏ Minimize interobserver variability
Secondary
Survey :
Outcome

Secondary
Survey :
Statistical
Analysis
High quality manuscript include
❏ A summary description of the
statistical tests used in the
study
❏ Comments should be made on
what assumptions and
statistical values were used to
determine the size of the
population studied

Secondary
Survey :
Statistical
Analysis
The studies on statistical point of
view
1. Purely describe an outcome -
Simplest
2. Compare outcomes- More
complex
3. Compare outcomes while
adjusting for potential
confounding variables- Most
complex

Articles on
methodology have
reviewed some of the
techniques used to
address this issue
Secondary
Survey :
Statistical
Analysis
* Singer AJ, Thode HC Jr, Hollander JE. Research fundamentals: selection and development of clinical outcome measures.
Acad Emerg Med. 2000 Apr. 7(4):397401. [Medline].
*Schriger DL. Problems with current methods of data analysis and reporting, and suggestions for moving beyond incorrect ritual.
Eur J Emerg Med. 2002 Jun. 9(2):2037. [Medline].

Secondary
Survey :
Results
❏ Best to begin by simply
scanning tables and
figures
❏ All graphic summaries
should be clearly labeled
and appropriately scaled
❏ Ideally, the text should
serve only to clarify these or
to point out highlights
❏ A figure or diagram that
depicts the flow of the
subjects studied is worth a
review.

Secondary
Survey :
Results
❏ Reader should have a clear
understanding of the
investigation and anticipate
presentation of certain
results
❏ A well written results section
first describes patients
involved in the study and
then determines whether
study groups were
sufficiently similar

Secondary
Survey :
Results
The reader should be
able
❏ To verify if any potential
confounding variables
were present
❏ It might affect a prognosis
or treatment outcome
❏ To establish that baseline
values of the outcome
index were similar among
comparison groups

Secondary
Survey :
Results
The reader should
check
❏ How many patients were
eligible for the study?
❏ How many were actually
enrolled into the protocol?
❏ How many were completed
it ?
❏ Were subjects who
withdrew clearly accounted
for ?

Secondary
Survey :
Results
The reader should
check
❏ Results of statistical
analysis should be
provided?
❏ All adverse outcomes
should be reported ?
❏ Are the outcomes reported
as p values or point
estimates?
❏ If the point estimates , are
confidence intervals
provided?

Secondary
Survey :
Results

Confidence interval (CI) is a type of
interval estimate, computed from the
statistics of the observed data, that
might contain the true value of an
unknown population parameter. ... Most
commonly, the 95%confidence level is
used

Secondary
Survey :
Discussion
It provides the authors an
opportunity
❏ To point out their most noteworthy
results
❏ To interpret results
❏ To explain their importance

❏ To compare their results with
previously reported studies
❏ To comment on similarities or
difference
❏ Allows them to cite limitations of
their study
❏ To suggest new directions for
appropriate research
Secondary
Survey :
Discussion

While reviewing an
article
❏ Achieving statistical significance
only minimizes the possibility that
results could have occurred by
chance alone
❏ It implies nothing about actual
importance or clinical significance
of the results
Secondary
Survey :
Discussion

While reviewing an article
❏ Large studies can achieve statistical
significance by demonstrating only
small clinical differences in
outcomes
❏ While not limited to the following,
limitations frequently revolve around
the issues of generalizability,
precision, bias, and confounding
Secondary
Survey :
Discussion

1. It must be consistent with the
study objective
2. It must be justified by the
study results
3. It should not over generalize
the results of the study
Secondary
Survey :
Conclusion

Do a quick check
❏ Review the study aim or objective
at the end of the introduction
❏ Examine whether the question
posed was answered by the
conclusion
Secondary
Survey :
Conclusion

How to review an article ?
Review Criteria
for Various
studies
Can the reader believe
the study results?
❏ Concepts of Validity.

❏ Reliability implies
consistency: if you take
the ACT five times, you
should get roughly the
same results every time
❏ A test is valid if it
measures what it’s
supposed to.
❏ Tests that are valid
are also reliable
❏ The ACT is valid (and
reliable) because it
measures what a
student learned
❏ However, tests that
are reliable aren’t
always valid

● Kuder-Richardson 20: a measure of
internal reliability for a binary test (i.e.
one with right or wrong answers).
● Cronbach’s alpha: measures internal
reliability for tests with multiple possible
answers
Review Criteria
for Various
studies
Tests of
Reliability

Review Criteria
for Various
studies
❏ Threats to a
study validity
Threats
❏ External
❏ Internal
Threats
❏ Controllable
❏ Non- Controllable

Review Criteria
for Various
studies
❏ External
threats to
validity
❏ Inability of study results to be
generalized to a population or
situation other than that
studied.
❏ Even if a study is internally valid
and the demonstrated outcome
real, results are not guaranteed
to be applicable to other
settings.
❏ Assess whether the methods
described in the setting or
circumstances under which the
clinical problem typically is
encountered.

Review Criteria
for Various
studies
❏ Internal
threats to
validity
Problems with
❏ Study design
❏ Implementation
❏ Bias is the systematic introduction of error,
which distorts results of a study in a
nonrandom way
❏ Most studies have some potential sources of
bias
Objective
facts
What confirm
Your beliefs
What
you
see

Review Criteria
for Various
studies
❏ Internal
threats to
validity

Review Criteria
for Various
studies
❏ Internal
threats to
validity
Bias
❏ Researchers are responsible for
understanding the influence bias
❏ They should minimize bias effects
when possible
❏ Identify potential bias when publishing
their results
❏ Bias is entirely different than
chance, which is a purely random
study outcome

❏ Demonstrated results are the
result of random chance rather than
to a real association or cause effect
relationship
❏ A primary purpose of statistical
analysis is to estimate the
likelihood that results obtained could
have occurred solely by chance.
Review Criteria
for Various
studies
❏ Internal
threats to
validity

❏ Significant effects on the
outcome of a study
❏ Very common research
problem
❏ Be alert for confounding
variables
❏ Authors may not consistently
identify them.
Review Criteria
for Various
studies
❏ Internal
threats to
validity
❏ Confounding
variables

Review Criteria
for Various
studies
❏ Internal
threats to
validity
❏ Confounding
variables

How to review an article ? The Hawthorne effect refers to
the inclination of some people to
work harder and perform better
when they are being observed as
part of an experiment
Review Criteria
for Various
studies
❏ Internal
threats to
validity
❏ Hawthorne
effect

❏ Physicians who know that they are
being measured may be influenced
to administer the drug earlier.
❏ This tendency of the study situation
to artificially influence the outcome of
the study is termed the Hawthorne
Effect.
Review Criteria
for Various
studies
❏ Internal
threats to
validity
❏ Hawthorne
effect

❏ A number of systems
exist for categorizing
types of studies
❏ Often overlap to some
degree
❏ Potentially causing the
reader confusion
Type of
studies by
Type of Design

Type of
studies by
Type of Design
Time frame
❏ Cross sectional
❏ Longitudinal

Type of
studies by
Type of Design
❏ Cross
sectional
❏ Observations made at one point
in time (a snapshot in time)
❏ Used for epidemiologic
purposes
❏ Used to establish a relationship or
association between two or
more variables
❏ Disadvantage - Cause and effect
cannot be determined

Type of
studies by
Type of Design
❏ Cross
sectional

❏ Observations over time
❏ Provide an opportunity for an
intervention
❏ Provide subsequent analysis
of cause and effect
Type of studies
by Type of
Design
❏ Longitudinal

Type of studies
by Type of
Design

Type of
studies by
Type of Design
Time frame
❏ Retrospective
❏ Prospective

❏ Collect data from written material
such as medical records created
before the study was designed or by
subject recall
❏ Verifying the existence of a risk
factor or outcome condition is
difficult
❏ Establishing a cause and effect
relationship is difficult
❏ Subject to recall bias as well as
selection bias
Type of studies
by Type of
Design
Retrospective
studies

❏ Suitable to study rare diseases
or conditions
❏ Useful to identify problems for
subsequent prospective trials
(Hypothesis generating)
Type of studies
by Type of
Design
Retrospective
studies

❏ Follow subjects forward in
time
❏ Collect data as they are
generated
❏ Interventional or
Observational
Type of studies
by Type of
Design
Prospective
studies

Begin with an
❏ Outcome of interest
❏ Exposure of interest
Type of
studies by
Type of Design

❏ Follow a cohort of subjects with
and without the exposure
❏ Usually prospective
❏ Eg: Motor Vehicle Collision
Type of studies
by Type of
Design
Exposure of
interest

❏ Identifying subjects with and
without a particular disease
(cases and controls)
❏ Often retrospective
Type of studies
by Type of
Design
Outcome of
interest

Type of studies
by Type of
Design
Interventional
design-
Controlled
❏ Intervention or control over
variables that can affect the
outcome
❏ Involves the evaluation of a
specific therapy administered
to each patient in a study group
❏ No intervention in another group
(Control group)
❏ Direct comparisons of outcome
❏ Assignment of patients are
Random

❏ The standard study
design for making
determinations of cause
and effect or the value
of a specific
intervention.
Type of studies by
Type of Design
❏ Interventional
design
❏ Prospective
Randomised
Controlled
Trail

Limitations
❏ Expense
❏ Ethical problems in testing
new therapies
❏ Ethical issues in
withholding them from
control group
❏ Time, money, and effort
necessary to perform the
study
Type of studies by
Type of Design
❏ Interventional
design
❏ Prospective
Randomised
Controlled
Trail

❏ Withholding a specific
intervention is not possible (eg,
if it is standard of care)
Type of studies
by Type of
Design
❏ Interventional
design-
Uncontrolled

❏ Do not involve analysis of
effects of an intervention
❏ Members of study group
share effects of a specific
characteristic
❏ No comparable control
group
❏ No comparison of effect
❏ Subjects are followed,
without intervention, with
respect to a particular
outcome
Type of studies by
Type of Design
❏ Interventional
design
❏ Prospective
Observational
Trail

Advantages
❏ Establish comparable
subjects prior to
beginning the study
❏ Ability to follow them over
time
❏ Important when risk
factors associated with a
particular disease or
condition
Type of studies by
Type of Design
❏ Interventional
design
❏ Prospective
Observational
Trail

Type of studies by
Content
❏ Most articles under
review can be placed
into one of the
following categories
1. Evaluation of a new therapy
2. Evaluation of a new diagnostic
test
3. Determination of the etiology of
a condition
4. Prediction of the outcome
5. Natural course of a condition
❖ Each categories has different criteria for scientific review

Scientific
review -
main issues
❏ Study on
New
therapy
Results 1. Magnitude of treatment effect
2. Precision of the measurement
Validity 1. Was there randomization into
treatment groups
2. Accounting for all study patients
3. Blinding of participants and personnel
4. Equality of treatment groups at
baseline
Impact 1. Applicability of results
2. Benefits versus risk and cost

How to review an
article ?
Scientific
review - main
issues
❏ Study on
New
diagnostic
test
Results 1. Presentation of likelihood ratios
2. Receiver operating characteristic (ROC)
curve
Validity 1. Was there blind and independent
comparison to an accepted criterion
("gold") standard
2. Was the test applied to a wide patient
spectrum
3. What was the effect of performing the
criterion standard
4. Ease of test replication
Impact 1. Patient applicability
2. Effect of clinical management of the
patient's condition
3. Effect on patient care

Receiver
Operating
Characteristic
(ROC) curve

Scientific
review - main
issues
❏ Study on
Etiology
Results 1. Strength of exposure to outcome
2. Precise risk estimate
Validity 1. Group similarity other than the
variable (exposure) of interest
2. Same exposure measurements
3. Strong temporal relationship
4. Adequate follow up
Impact 1. Results apply to patient population
2. Magnitude of risk

Scientific
review - main
issues
❏ Study on
Prediction
of
outcome
Results 1. Magnitude of outcome likelihood
2. Precision of likelihood estimate
Validity 1. Representative patient sample
2. Sufficient follow up
3. Use of unbiased
4. Objective outcome criteria
Impact 1. Equivalent patient population for
comparison
2. Effect on therapy choice

❏ Reviewing the medical literature poses
a challenge to the busy emergency
physician
❏ A willingness and ability to do so
enhance the quality of the practice
❏ A brief primary survey of the article of
interest informs the reader as to the
potential value of the findings and to
whether a more indepth review is
indicated

❏ Detailed analysis (secondary survey) allows
the reader to determine whether the article's
conclusion is supported by its results and
whether these results are believable
❏ Knowledge of the standard anatomy of an
article and the idiosyncrasies of the various
types of studies will assist the reader to
intelligently review the medical literature
efficiently

How to select , read & review an article

More Related Content

What's hot (20)

Similar to How to select , read & review an article (20)

More from Dr.Venugopalan Poovathum Parambil (20)

Recently uploaded (20)

How to select , read & review an article