Presentation Sioo Evidence-Based Practice (Dutch)

Evidence-Based Practice
Sioo, 8 april 2013
Utrecht

Evidence-based practice

 Wat is het?
 Waar komt het vandaan?
 Hoe ziet het er uit in een opleiding?

Evidence based practice:
Wat is het?

Definition


Improve information to support decision making


Uitgangspunt bij evidence-based practice is
dat beslissingen gebaseerd dienen te zijn
op een combinatie van kritisch denken en
de best beschikbare 'evidence'.

Evidence based practice

Met het begrip 'evidence' wordt niet
meer bedoeld dan 'informatie'.

Dit kan informatie zijn afkomstig uit
wetenschappelijk onderzoek, maar ook
interne bedrijfsinformatie en persoonlijke
ervaring geldt als 'evidence’.


In principe neemt iedere manager dus
beslissingen op basis van 'evidence'.

De meeste managers besteden echter
nauwelijks aandacht aan de kwaliteit
van de 'evidence' waarop ze hun
beslissingen baseren.

Evidence-based practice:

 kritisch denken
 van verschillende informatiebronnen
gebruik maken >> best available evidence?

 de evidence kritisch tegen het licht houden
 denken in termen van waarschijnlijkheid
in plaats van 'golden bullets'.


Best available Professional
external evidence expertise and
judgment

Evidence-based
decision

Best available Stakeholders’
internal evidence values and concerns

Waar komt het vandaan?

What field is this?

 “there is a large research-user gap”
 “practitioners do not read academic journals”
 “the findings of research into what is an effective intervention
are not being translated into actual practice”
 “academics not practitioners are driving the research agenda”
 “the relevance, quality and applicability of research is
questionable”
 “practice is being driven more by fads and fashions than
research”
 “many practices are doing more harm than good”

Medicine: Founding fathers

David Sackett Gordon Guyatt

McMaster University Medical School, Canada

Problem I: persistent convictions

if you’re
breathe into a bag
hyperventilating


elderly people who have give them a drug that
an irregular heartbeat are reduces the
much more likely to die of number of
coronary disease irregular beats

How 40,000 cardiologists can be wrong

In the early 1980s newly introduced
anti-arrhythmic drugs were found to be
highly successful at suppressing
arrhythmias.

Not until a RCT was performed was it
realized that, although these drugs
suppressed arrhythmias, they actually
increased mortality.

By the time the results of this trial were
published, at least 100,000 such
patients had been taking these drugs.

Problem II: too much information

 More than 1 million articles in 40,000 medical journals per
year (= 1995; now probably more than 2 million). For a
specialist to keep up this means reading 25 articles every
day (for a GP more than 100!)


Maslow, A.H. (1943). "A Theory of Human Motivation," Psychological Review 50(4)

Wahba, M. A., & Bridwell, L. G. (1976). Maslow reconsidered: A review of
research on the need hierarchy theory. Organizational Behavior and Human
Performance, 15(2)

True or false?

1. Incompetent people benefit more from feedback than
highly competent people.

2. Task conflict improves work group performance while
relational conflict harms it.

3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.

How evidence-based are we?

HR Professionals' beliefs about effective human resource practices: correspondence
between research and practice, (Rynes et al, 2002, Sanders et al 2008)

 959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body of evidence
 true / false / uncertain

On average: 35% - 57% correct

Problem II: too much information

 HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR
manager to keep up this means reading 3 to 4 articles
every day (for a „general‟ manager more than 50!)

BTW: most of the research is seriously
flawed or irrelevant for practice

David Sackett
 Half of what you learn in medical school will be
shown to be either dead wrong or out-of-date
within 5 years of your graduation; the trouble is that
nobody can tell you which half.

 The most important thing to learn is how to learn
on your own: search for the evidence!

 (Remember that your teachers are as full of bullshit
as your parents)

The 5 steps EBP: pull approach

1. Formulate a focused question (Ask)

2. Search for the best available evidence (Acquire)

3. Critically appraise the evidence (Appraise)

4. Integrate the evidence with your professional
expertise and apply (Apply)

5. Monitor the outcome (Assess)

Evidence-Based Practice

1991 Medicine
1998 Education
1999 Social care, public policy
2000 Nursing
2000 Criminal justice
???? Management?

Hoe ziet het er uit in de opleiding?


Fase 1: Het ontwikkelen van kritisch en wetenschappelijk
denken dat leidt tot een professioneel-kritische attitude met
betrekking tot organisatievraagstukken

Fase 2: Het kunnen formuleren van een expliciete vraag en
op basis van deze vraag kunnen zoeken in online databases
naar uitkomst van relevant wetenschappelijk onderzoek.

Fase 3: Het kritisch kunnen beoordelen van
wetenschappelijke en organizational evidence (critical
appraisal)

Fase 4: Uitkomst van wetenschappelijk onderzoek kunnen
toepassen in de eigen beroepspraktijk (application of science)


scientific evidence expertise and
judgment

Evidence-based
decision

organizational
values and concerns
evidence

Trust me, 20 years of experience

Het feilbare brein

System 1
 Snel, actie
 Intuitief, associatief
 shortcuts & biasses

System 2
 Langzaam (lui!)
 Rationeel
 Nadenken

Bounded rationality

neo cortex
(systeem 2)

limbische systeem
en hersenstam
(systeem 1)

Systeem 1: het feilbare brein

 Seeing order in randomness
 Mental corner cutting
 Misinterpretation of incomplete data
 Halo effect
 False consensus effect
 Group think  Confirmation bias
 Self serving bias  Authority bias
 Sunk cost fallacy  Small numbers fallacy
 Cognitive dissonance reduction  In-group bias
 Recall bias
 Anchoring bias
 Inaccurate covariation detection
 Distortions due to plausibility

Fase 1: Kritisch & wetenschappelijk denken

1. Denkfouten

2. Informatiebronnen

3. Mythbusting

4. Assumpties

Richard Feynman

“The first principle is that you must not fool
yourself - and you are the easiest person to
fool”.

Seeing order in randomness

 Een Type I fout of een vals positief:
denken dat er een patroon / verband is
terwijl dat er in het echt niet is.

 Een Type II fout of een vals negatief:
denken dat er geen patroon / verband is
terwijl dat er in het echt wel is
Dr. Michael Shermer
(Director of the Skeptics Society)

Het feilbare brein: patern recognition

 Een Type I fout of een vals positief: denken dat het
geritsel in de bosjes een gevaarlijk roofdier is, terwijl
het gewoon de wind is (goedkoop foutje)


 Een Type II fout of een vals negatief: denken dat het
geritsel in de bosjes gewoon de wind is, terwijl het een
gevaarlijk roofdier is (duur foutje)


 Een Type I fout of een vals positief: denken dat het
geritsel in de bosjes een gevaarlijk roofdier is, terwijl
het gewoon de wind is (goedkoop foutje)

 Een Type II fout of een vals negatief: denken dat het
geritsel in de bosjes gewoon de wind is, terwijl het een
gevaarlijk roofdier is (duur foutje)


Ook ervaren mensen en experts zien patronen
en verbanden waar ze niet zijn.

stress & lifestyle peptic ulcer

Oct 2005

Peptic ulcer – an infectious disease!
This year's Nobel Prize in Physiology or Medicine goes to Barry Marshall and Robin
Warren, who with tenacity and a prepared mind challenged prevailing dogmas. By
using technologies generally available (fibre endoscopy, silver staining of
histological sections and culture techniques for microaerophilic bacteria), they
made an irrefutable case that the bacterium Helicobacter pylori is causing disease.
By culturing the bacteria they made them amenable to scientific study.

In 1982, when this bacterium was discovered by Marshall and Warren, stress and
lifestyle were considered the major causes of peptic ulcer disease. It is now
firmly established that Helicobacter pylori
causes more then 90% of duodenal ulcers.
The link between Helicobacter pylori
infection and peptic ulcer disease has been
established through studies of human
volunteers, antibiotic treatment studies and
epidemiological studies.

Errors and Biases of Human Judgment

superstitious superstitious
rituals rituals

more stress = more prone to biases

Errors and Biases of Human Judgment

Presentation Sioo Evidence-Based Practice (Dutch)

Beliefs vs Evidence

“What gets us into trouble is not what we
don't know,

it's what we know for sure that just isn't so.”

Mark Twain


Ook slimme mensen houden er verkeerde ideeën op
na, niet omdat ze dom of eigenwijs zijn, maar omdat het
de meest logische conclusie is op basis van hun eigen
ervaringen.

(systeem 1 doet altijd mee!)

The 5 steps EBP

1. Formulate a focused question (Ask)

2. Search for the best available evidence (Acquire)

3. Critically appraise the evidence (Appraise)

4. Integrate the evidence with your professional
expertise and apply (Apply)

5. Monitor the outcome (Assess)

Focused question?

 Does team-building work?
 What are the costs and benefits of self-steering teams?
 What are the success factors for culture change?
 Does management development improve the
performance of managers?

 Does employee participation prevent resistance to
change?

 How do employees feel about 360 degree feedback?

Foreground question?

 Does team-building work?

 What is a „team‟?
 What kind of teams?
 In what contexts/settings?
 What counts as „team-building‟?
 What does „work‟ mean?
 What outcomes are relevant?
 Over what time periods?

Answerable question: PICOC

P = Population

I = Intervention or success factor

C = Comparison

O = Outcome

C = Context

2. Finding the best available evidence

Searching evidence

What do we search?

What do we search?

Current Information

Overview of a subject

General background

Academic Information

Statistical Information

Theories about a subject

Company information

Databases

 ABI/INFORM
 Business Source Elite
 PsycINFO
 Web of Knowledge
 ERIC
 Google Scholar

Searching evidence

How do we search?

Search Strategy

Search strategy

Two types of search strategies

Snowball method Building blocks method

Question

Imagine you are a consultant, your client is the board of
directors of a large Canadian health-care organization.
The board of directors has plans for a merger with a
smaller healthcare organization in a nearby town.
However, it‟s been said that the organizational culture
differs widely between the two organizations. The board
of directors asks you if this culture-difference can impede
a successful outcome of a merger. Most of them
intuitively sense that cultural differences matter, but they
want an evidence-based advice.


What else would you like to know?


P: What kind of Population are we talking about? Middle managers,
back-office employees, medical staff, clerical staff?

O: What kind of Outcome are we aiming for? Employee productivity,
return on investment, profit margin, competitive position, innovation
power, market share, customer satisfaction?

P/C: And how is the assumed cultural difference assessed? Is it the
personal view of some managers or is it measured by a validated
instrument?


According to the board the objective of the merger is to
integrate the back-office of the two organizations (ICT,
finance, purchasing, facilities, personnel administration,
etc.) in order to create economy of scale. The front
offices and primary process of the two organizations will
remain separate.

The cultural difference is not objectively assessed (it is
the perception of the senior managers of both
organizations).


P = back office employees in a healthcare organisation

I = merger, integration back office

C = status quo

O = economy of scale

C = different organizational culture, unequal

What is the best research design?

 Randomized controlled study?
 Grounded theory approach?
 Cohort / panel study?
 Qualitative field research?
 Longitudinal study?
 Post-test only study?
 Survey?
 Action research?
 Case study?

What is the best design?

quants vs quallies, positivists vs post structuralist, etc

Research designs

Which design for which question?

Types of questions

Does it work?

Does it work better than ....?

Does it have an effect on ....?
Effect
What is the success factor for ....?

What is required to make it work ...?

Will it do more good than harm?

Types of questions: non-effect

Needs: What do people want or need?

Attitude: What do people think or feel?

Experience: What are peoples’ experiences?

Prevalence: How many / often do people / organizations ...?

Procedure: How can we implement ...?

Process: How does it work?

Explanation: Why does it work?

Economics: How much does it cost?

Internal validity

internal validity = indicates to what extent the
results of the research may be biased and is thus
a comment on the degree to which alternative
explanations for the outcome found are possible.

Causal relations

We are pattern seeking primates:
we are predisposed to see order
and causal relations in the world

Causality

When do we know there is a causal relation?

1. Are the "cause" and the "effect” related?
measurements, effect size

1. Does the "cause" precede the "effect" in time?
before and after measurement

2. Are there no plausible alternative explanations for
the observed effect?
randomization, control group

Bias & Confounding

Research shows:

Shoe size > quality of handwriting

Smoking youngsters > better lung function

Which design for which question?

Explanation

Different types of research questions
require different types of research designs,

but ...

Step 3: Critical appraisal of studies

How to read a research article?

Critical appraisal: quick and dirty

Is the study design appropriate to the stated
aims?

Are the measurements likely to be valid and
reliable?

Was there a relevant effect size?

Is the outcome (population, type of organization)
generalizable to your situation?

Appraisal

Critical appraisal questionnaires

www.cebma.org/ebp-tools

Aantal te laat betalers neemt toe

scientific evidence expertise and
judgment

Evidence-based
decision

organizational
values and concerns
evidence

CAT: Critically Appraised Topic

CAT: structure

1) Background / context

2) Question (PICOC)

3) Search strategy

4) Results / evidence summary max 3 pag.
5) Findings

6) Limitations

7) Recommendation

Presentation Sioo Evidence-Based Practice (Dutch)

More Related Content

What's hot (20)

Similar to Presentation Sioo Evidence-Based Practice (Dutch) (20)

More from Center for Evidence-Based Management (18)

Recently uploaded (20)

Presentation Sioo Evidence-Based Practice (Dutch)