The advanced handbook of methods in evidence based healthcare Brazier

The advanced handbook of methods in evidence
based healthcare Brazier download
https://ebookgate.com/product/the-advanced-handbook-of-methods-
in-evidence-based-healthcare-brazier/
Get Instant Ebook Downloads – Browse at https://ebookgate.com

Get Your Digital Files Instantly: PDF, ePub, MOBI and More
Quick Digital Downloads: PDF, ePub, MOBI and Other Formats
Evidence Based Medicine and the Changing Nature of
Healthcare 2007 IOM Annual Meeting Summary Learning
Healthcare Systems 1st Edition Mark B. Mcclellan
https://ebookgate.com/product/evidence-based-medicine-and-the-
changing-nature-of-healthcare-2007-iom-annual-meeting-summary-
learning-healthcare-systems-1st-edition-mark-b-mcclellan/
Handbook of Evidence Based Veterinary Medicine 1st
Edition Peter Cockcroft
https://ebookgate.com/product/handbook-of-evidence-based-
veterinary-medicine-1st-edition-peter-cockcroft/
Evidence Based Educational Methods Educational
Psychology 1st Edition Daniel J. Moran
https://ebookgate.com/product/evidence-based-educational-methods-
educational-psychology-1st-edition-daniel-j-moran/
Evidence Based Pediatric Oncology 2nd ed Evidence Based
2nd Edition Ross Pinkerton
https://ebookgate.com/product/evidence-based-pediatric-
oncology-2nd-ed-evidence-based-2nd-edition-ross-pinkerton/

Dynamic Capacity Management for Healthcare Advanced
Methods and Tools for Optimization 1st Edition Pierce
Story
https://ebookgate.com/product/dynamic-capacity-management-for-
healthcare-advanced-methods-and-tools-for-optimization-1st-
edition-pierce-story/
Evidence Based To Value based Medicine 1st Edition
Brown
https://ebookgate.com/product/evidence-based-to-value-based-
medicine-1st-edition-brown/
Evidence Base of Clinical Diagnosis Evidence Based 1st
Edition J. André Knottnerus
https://ebookgate.com/product/evidence-base-of-clinical-
diagnosis-evidence-based-1st-edition-j-andre-knottnerus/
Nursing Research Methods and Critical Appraisal for
Evidence Based Practice 8th Edition Geri Lobiondo-Wood
https://ebookgate.com/product/nursing-research-methods-and-
critical-appraisal-for-evidence-based-practice-8th-edition-geri-
lobiondo-wood/
Gynaecology Evidence Based Algorithms Jyotsna Pundir
https://ebookgate.com/product/gynaecology-evidence-based-
algorithms-jyotsna-pundir/

The Advanced Handbook of
Methods in Evidence
Based Healthcare
edited by
Andrew Stevens, Keith Abrams,
John Brazier, Ray Fitzpatrick
and Richard Lilford
eBook covers_pj orange.indd 58 21/2/08 2:09:31 pm

METHODS IN
EVIDENCE BASED
HEALTHCARE

The advanced handbook of methods in evidence based healthcare Brazier

METHODS IN
EVIDENCE BASED
HEALTHCARE
Edited by
A N D R E W S T E V E N S
K E I T H A B R A M S
J O H N B R A Z I E R
R A Y F I T Z P A T R I C K
and R I C H A R D L I L F O R D
SAGE Publications
London • Thousand Oaks • New Delhi

Editorial arrangement, Introduction and Part Introductions
© Andrew Stevens, Keith R. Abrams, John Brazier,
Ray Fitzpatrick and Richard J. Lilford 2001
Chapters 1–27 © the contributors 2001
First published 2001
Apart from any fair dealing for the purposes of research or
private study, or criticism or review, as permitted under the
Copyright, Designs and Patents Act, 1988, this publication
may be reproduced, stored or transmitted in any form, or by
any means, only with the prior permission in writing of the
publishers, or in the case of reprographic reproduction, in
accordance with the terms of licences issued by the
Copyright Licensing Agency. Inquiries concerning
reproduction outside those terms should be sent to
the publishers.
SAGE Publications Ltd
6 Bonhill Street
London EC2A 4PU
SAGE Publications Inc
2455 Teller Road
Thousand Oaks, California 91320
SAGE Publications India Pvt Ltd
32, M-Block Market
Greater Kailash – I
New Delhi 110 048
British Library Cataloguing in Publication data
A catalogue record for this book is available from
the British Library
ISBN 0 7619 6144 5
Library of Congress catalog record available
Typeset by Photoprint, Torquay, Devon
Printed in Great Britain by The Cromwell Press Ltd,
Trowbridge, Wiltshire
Reprinted 2001

Contents
Contributors ix
Introduction
Methods in Evidence Based Healthcare and Health Technology
Assessment: An Overview 1
Andrew Stevens, Ray Fitzpatrick, Keith Abrams, John Brazier
and Richard Lilford
Part I CLINICAL TRIALS
Introduction 7
Richard Lilford and Andrew Stevens
1 Ethical Issues in the Design and Conduct of Randomised
Controlled Trials 11
Richard J. Lilford, Sarah J.L. Edwards, David A. Braunholtz,
Jennifer Jackson, Jim Thornton and Jenny Hewison
2 Ethics of Clinical Trials: Social, Cultural and Economic Factors 25
Richard E. Ashcroft
3 Factors that Limit the Number, Progress and Quality of
Randomised Controlled Trials: A Systematic Review 38
Sue Ross, Carl E. Counsell, William J. Gillespie, Adrian M. Grant,
Robin J. Prescott, Ian T. Russell, Iain R. Colthart, Sandra Kiauka,
Daphne Russell and Sue M. Shepherd
4 Results of Clinical Trials and Systematic Reviews:
To Whom Do They Apply? 56
Dianne O’Connell, Paul Glasziou, Suzanne Hill,
Jasminka Sarunac, Julia Lowe and David Henry
5 The Placebo Effect: Methodological Process and Implications
of a Structured Review 73
Rosemary Crow, Heather Gage, Sarah Hampson, Jo Hart,
Alan Kimber and Hilary Thomas

Part II OBSERVATIONAL AND QUALITATIVE
METHODS
Introduction 91
Ray Fitzpatrick
6 Randomised and Non-Randomised Studies: Threats to Internal
and External Validity 95
Colin Sanderson, Martin McKee, Annie Britton, Nick Black,
Klim McPherson and Chris Bain
7 A Review of Observational, Quasi-Experimental and Randomised
Study Designs for the Evaluation of the Effectiveness of
Healthcare Interventions 116
Barnaby C. Reeves, Rachel R. MacLehose, Ian M. Harvey,
Trevor A. Sheldon, Ian T. Russell and Andrew M.S. Black
8 The Potential Use of Routine Datasets in Health Technology
Assessment 136
James Raftery, Andrew Stevens and Paul Roderick
9 Using Routine Data to Complement and Enhance the Results
of Randomised Controlled Trials 149
James D. Lewsey, Gordon D. Murray, Alastair H. Leyland and
F. Andrew Boddy
10 Qualitative Methods in Health Technology Assessment 166
Elizabeth Murphy and Robert Dingwall
Part III MEASUREMENT OF BENEFIT AND COST
Introduction 179
John Brazier
11 Criteria for Assessing Patient Based Outcome Measures for Use
in Clinical Trials 181
Ray Fitzpatrick, Claire Davey, Martin J. Buxton and
David R. Jones
12 The Use of Health Status Measures in Economic Evaluation 195
John Brazier, Mark Deverill and Colin Green
13 Collecting Resource Use Data for Costing in Clinical Trials 215
Katharine Johnston, Martin J. Buxton, David R. Jones and
Ray Fitzpatrick
vi The Advanced Handbook of Methods in Evidence Based Healthcare

14 Eliciting Time Preferences for Health 231
John Cairns and Marjon van der Pol
15 The Conduct and Design of Questionnaire Surveys in
Healthcare Research 247
Elaine McColl, Ann Jacoby, Lois Thomas, Jennifer Soutter,
Claire Bamford, Nick Steen, Roger Thomas, Emma Harvey,
Andrew Garratt and John Bond
Part IV ANALYTICAL METHODS
Introduction 273
Keith R. Abrams
16 Bayesian Methods 275
David J. Spiegelhalter, Jonathan P. Myles, David R. Jones and
Keith R. Abrams
17 Methods for Evaluating Organisation- or Area-Based Health
Interventions 295
Martin Gulliford, Obioha C. Ukoumunne, Susan Chinn,
Jonathan Sterne, Peter Burney and Allan Donner
18 Handling Uncertainty in Economic Evaluation 314
Andrew H. Briggs and Alastair M. Gray
19 A Review of the Use of the Main Quality of Life Measures,
and Sample Size Determination for Quality of Life Measures,
Particularly in Cancer Clinical Trials 338
Michael J. Campbell, Steven A. Julious, Sarah J. Walker,
Steve L. George and David Machin
20 Simultaneous Analysis of Quality of Life and Survival Data 352
Lucinda J. Billingham, Keith R. Abrams and David R. Jones
Part V CONSENSUS, REVIEWS AND META-ANALYSIS
Introduction 367
Andrew Stevens and Keith R. Abrams
21 Publication and Related Biases 371
Fujian Song, Alison Eastwood, Simon Gilbody, Lelia Duley and
Alex Sutton
vii
Contents

22 Meta-Analysis in Health Technology Assessment 391
Alexander J. Sutton, David R. Jones, Keith R. Abrams,
Trevor A. Sheldon and Fujian Song
23 Assessing the Quality of Reports of Randomised Trials Included
in Meta-Analyses: Attitudes, Practice, Evidence and Guides 409
David Moher, Terry P. Klassen, Alison L. Jones, Ba’ Pham,
Deborah J. Cook, Alejandro R. Jadad, Peter Tugwell and
Michael Moher
24 Consensus Development Methods, and their Use in Creating
Clinical Guidelines 426
Nick Black, Maggie Murphy, Donna Lamping, Martin McKee,
Colin Sanderson, Janet Askham and Theresa Marteau
Part VI IDENTIFYING AND FILLING GAPS IN
THE EVIDENCE
Introduction 449
Andrew Stevens
25 Identifying New Healthcare Technologies 451
Glenn Robert, Andrew Stevens and John Gabbay
26 Timing of Assessment of Fast-Changing Health Technologies 471
Graham Mowatt, Adrian M. Grant, D. Jane Bower,
John A. Brebner, John A. Cairns and Lorna McKee
27 Preliminary Economic Evaluation of Health Technologies 485
Gareth Harper, Joy Townsend and Martin Buxton
Index 499
viii The Advanced Handbook of Methods in Evidence Based Healthcare

Contributors
Keith Abrams (editor) is Reader in Medical Statistics, in the Department of
Epidemiology and Public Health, at the University of Leicester, having
previously held posts at King’s College School of Medicine and Dentistry and
the London School of Hygiene and Tropical Medicine. He has gained degrees
from the Universities of Warwick, Leicester and Liverpool. His research
interests include: the development and application of Bayesian methods in
Health Services Research (HSR)/Health Technology Assessment (HTA),
systematic reviews and meta-analysis, and analysis of time to event data. He
is a fellow of the Royal Statistical Society and a Chartered Statistician.
Richard Ashcroft is Lecturer in Medical Ethics at Imperial College School
of Medicine. He trained as an undergraduate in mathematics and philosophy at
Cambridge University, before completing a PhD in ethics and philosophy of
science, also at Cambridge. The chapter in this volume is based on work
carried out as a postdoctoral research fellow in the Philosophy Department at
Liverpool University, funded by the NHS Health Technology Assessment
programme, grant 93/41/4, held by Dr Jane Hutton with Prof. David Chadwick,
Prof. Stephen Clark, Prof. Richard Edwards, and Ms. Lucy Frith. He is
currently working on ethical issues in medical research and technology
assessment.
Janet Askham is Professor of Gerontology and Director of the Age Concern
Institute of Gerontology at King’s College London. She is a sociologist
whose research career has covered topics spanning the life course: fertility
behaviour, marriage and family relationships, chronic illness and disability,
care in old age. She has particular interests in family life and care for people
with dementia, late-life marriage and inter-generational relationships.
Chris Bain is Reader in Social and Preventive Medicine at the University of
Queensland. Whilst much of his research is in cancer epidemiology, he has
maintained an interest in health services research which was happily fanned
into life during a sabbatical spent with his co-authors.
Claire Bamford graduated in Psychology from the University of London and
also has an MSc in social research methods from the University of Surrey.

Her research interests include provision of services and informal care to older
people with dementia, and the measurement of outcomes of community care.
She is currently a research fellow at the Social Policy Research Unit at the
University of York working on a Department of Health-funded programme of
research into ways of collecting and using information on the outcomes of
community care in routine practice.
Cindy Billingham currently works at the Cancer Research Campaign Trials
Unit at the University of Birmingham as a Senior Biostatistician. Her area of
special interest is the analysis of quality of life data, and she is registered for
a part-time PhD at Leicester University researching into joint modelling of
quality of life and survival data.
Andrew Black was educated in Pakistan, Scotland and Oxford. He did his
Postgraduate clinical training in Oxford, Gloucester, Sydney and Auckland.
He was a Lecturer in Anaesthesia at the University of Oxford from 1977–1982.
Since then he has been a Consultant Senior Lecturer in Anaesthesia at the
University of Bristol. His academic interests are in pulmonary gas exchange,
control of breathing, postoperative analgesia, clinical applications of classifica-
tion approaches including statistical multi-variable and multi-variate modelling
and neural networks, applications of medical simulation and role play in
medical undergraduate education.
Nick Black is Professor of Health Services Research (since 1995) and Head
of the Department of Public Health and Policy at the London School of
Hygiene and Tropical Medicine (since May 1998). After qualifying in
medicine from Birmingham University in 1974, he worked in NHS hospitals
for two-and-a-half years before joining Save the Children Fund and running a
child health programme in Nepal for 18 months. He then underwent post-
graduate training in public health at Oxford including a doctorate on reasons
for the epidemic in surgery for glue ear in the UK. The next three years were
spent half time as a lecturer at the Open University writing a new distance-
learning course ‘Health and Disease’ with a biologist, sociologist and
economist, and half time as a Consultant in Public Health for Oxfordshire
Health Authority. In 1985 he moved to a Senior Lectureship at the LSHTM.
His main research interests are the evaluation and audit of health services
(particularly in the field of surgery), non-randomised methods of evaluation,
R&D policy, and organisational research. He is involved in several aspects of
the NHS R&D programme at national and regional levels. In 1996, together
with Nick Mays (a Visiting Professor in PHP), he founded the Journal of
Health Services Research & Policy. He was promoted to a Readership in 1993
and to his current Chair in 1995.
Andrew Boddy is an Honorary Senior Research Fellow in the Department of
Public Health, University of Glasgow and was formerly Director of the
University’s Public Health Research Unit. He has a long-standing interest in
x The Advanced Handbook of Methods in Evidence Based Healthcare

the uses of routinely assembled data for health services planning and
evaluation, including methods of their analysis and interpretation.
John Bond is a Sociologist with a special interest in social gerontology and
health services research. He is currently Professor of Health Services
Research at the Centre for Health Services Research at the University of
Newcastle upon Tyne and Associate Director at the Institute for the Health of
the Elderly, University of Newcastle upon Tyne. His research interests are
health policy and older people, in particular looking at social protection
policy for dependency in old age; health services research and older people
with dementia; health services research and older people who fall; disability
and older people; and Health Technology Assessment with particular refer-
ence to older people. He is a Member of the Health Technology Assessment,
Primary and Community Care Panel and the Health Technology Assessment
Commissioning Board.
D. Jane Bower currently holds a Chair in the Department of Economics and
Enterprise at Glasgow Caledonian University. She graduated with a PhD in
Molecular Genetics from Edinburgh University and worked initially for 15
years as a biomedical scientist at Stanford Medical School, Edinburgh
University and the MRC Human Genetics Unit. Since 1989 her research has
focused on issues surrounding the development of new technologies into
useful application. In the course of this research she has worked with public
and private sector organisations in Europe, the USA and Japan.
David Braunholtz is currently Senior Research Fellow in Medical Statistics
in the Department of Public Health and Epidemiology at the University of
Birmingham, having worked previously in medical schools in Leeds and
Managua, at the Greater London Council, and in the Scottish Office. He
graduated in Maths and Physics. Interests include research ethics, clinical
trials and Bayesian methods.
John Brazier (editor) is Professor of Health Economics and Director of the
Sheffield Health Economics Group (which includes 14 health economists) in
the School of Health and Related Research at the University of Sheffield.
His main interests have been in the measurement and valuation of health
services outcomes and economic evaluation, including the development of a
preference-based measure from the SF-36 for use in economic evaluation. He
continues to be actively involved in the design, analysis and interpretation of
economic evaluations conducted alongside clinical trials and modelling.
John Brebner is a Senior Lecturer in the Department of General Practice and
Primary Care at the University of Aberdeen; Director of the Remote Health
Care Unit/Telemedicine Laboratory and Visiting Professor at the United Arab
Emirates University. His main research interests and projects are the human
xi
Contributors

resource implications for the nursing profession in developing telemedicine
within the NHS; when and how to assess fast-changing technology; an evalu-
ation of the feasibility and diagnostic acceptability of tele-ultrasonography for
rural areas and telemedicine in Primary Care.
Andrew Briggs is a Joint MRC/South East Region Training Fellow in Health
Services Research at the Health Economics Research Centre, University of
Oxford. Andrew joined the Health Economics Research Centre in 1996. He is
particularly interested in statistical issues associated with economic evalu-
ation alongside clinical trials, and currently holds a joint MRC/South East
Region Training Fellowship to study statistical methods in cost–effectiveness
analysis. He also has a general interest in decision analysis and modelling
techniques applied to the economic evaluation of healthcare interventions,
and has been involved with economic evaluations of interferon treatment for
hepatitis, total hip replacement, and various screening programmes.
Annie Britton is a lecturer at the London School of Hygiene and Tropical
Medicine. She has worked in public health research for several years, and her
study interests include trial methodology, health promotion evaluation, and
most recently an epidemiological assessment of alcohol consumption in
Europe. She has published over 20 articles in peer-reviewed journals.
Peter Burney is Professor of Public Health Medicine and Chairman of the
Division of Primary Care and Public Health Sciences at King’s College
London. He is the Honorary Director of the Department of Health’s Social
Medicine and Health Services Research Unit. His current work is focused on
the aetiology and management of allergic and respiratory conditions.
Martin Buxton is Director of the Health Economics Research Group
(HERG) at Brunel University. He has been concerned with the economics of
healthcare and with health service policy for over 20 years. Martin has been
responsible for a number of major projects for the UK Department of Health,
including the early evaluation of the costs and benefits of the UK heart
transplant programmes. HERG’s recent work for the Department includes
research on liver transplantation, an evaluation of picture archiving and
communication systems, on screening programmes, and a project devising a
methodology to assess the payback on Health Services Research. He is
involved in many clinical trials, particularly of cardiac interventions. Pro-
fessor Buxton currently sits on a number of advisory groups: he is a member
of the NHS Standing Group on Health Technology, Chair of the NHS R&D
Methodology Group and a member of the HEFCE RAE 2001 Panel for
Community-Based Clinical Subjects.
John Cairns is an economist who has specialised in the area of health
economics for the past ten years. He is currently acting director of the Health
Economics Research Unit at the University of Aberdeen.
xii The Advanced Handbook of Methods in Evidence Based Healthcare

Michael Campbell is Professor of Medical Statistics with an interest in
Primary Care at the University of Sheffield. He is interested in the design and
analysis of trials, particularly in primary care. He is interested in sample size
calculations and analysis of quality of life data. He is a co-author with David
Machin of Statistical Tables for the Design of Clinical Trials (Blackwells) and
Medical Statistics: A Common Sense Approach (Wiley).
Susan Chinn is a Reader in Medical Statistics at the Department of Public
Health Sciences, King’s College, London. Her current work is mainly in
respiratory epidemiology, especially asthma, and she has been involved in
child growth surveillance for many years. She is interested in all applications
of statistics that further excellence in medical research, of which the design of
cluster randomised trials is a good example.
Iain Colthart is currently employed as a research and information officer at
the British Medical Association in Edinburgh. He contributed to the chapter
in this volume when he was employed as a research associate in the Medical
Statistics Unit at the Department of Public Health, University of Edinburgh.
His qualifications are B.Com. (Hons) and M.A.
Deborah Cook is an Associate Professor at McMaster University, and a
general internist, intensivist and epidemiologist. She is Director of the
Academic General Internal Medicine Fellowship at McMaster, Chair of the
Canadian Critical Care Trials Group and Consulting Editor for Journal of the
American Medical Association. Her research interests are in pneumonia,
gastrointestinal bleeding, end-of-life decisions and evidence based medicine.
Carl Counsell is a Specialist Registrar in neurology at the Western General
Hospital in Edinburgh with research interests in stroke, movement disorders
and epidemiology, particularly clinical trials and systematic reviews. He has
an MSc in epidemiology, and was involved in setting up the Cochrane Stroke
Group.
Rosemary Crow is Professor of Nursing Science in the European Institute of
Health and Medical Sciences at the University of Surrey. She trained as a
nurse before reading psychology at the University of Edinburgh. Her research
interests lie in clinical judgement within the field of medical and surgical
nursing.
Claire Davey obtained her Bachelor of Applied Science at the University of
Victoria, Australia. She worked in health promotion for three years before
moving to the University of Oxford, England in 1992 to work on research
into measurement of patient-assessed outcomes and methods of providing
information for cancer screening programmes. She is the co-author with Ray
Fitzpatrick, Martin Buxton and David Jones of a monograph published by the
Health Technology Assessment (HTA) Programme, Evaluating patient-based
xiii
Contributors

outcome measures for use in clinical trials, HTA Programme, University of
Southampton, 1998.
Mark Deverill is a research fellow in the Sheffield Health Economics Group
at the University of Sheffield. His main research interest lies in the use of
quality of life instruments in the economic evaluation of health technologies.
With colleagues he has worked on the SF-6D, the generation of utility values
from the SF-36.
Robert Dingwall is Professor of Sociology and Director of the Genetics and
Society Unit at the University of Nottingham. His career has spanned both
medical sociology, the field of his PhD from the University of Aberdeen, and
law and society studies, through work at the University of Oxford. These are
united by the application of qualitative research methods to questions about
professions, work, organisations and social interaction. He is currently
developing a new research and graduate programme for the study of social,
legal and ethical aspects of new genetic science and technologies as applied
to humans, plants and animals.
Allan Donner is Professor and Chair in the Department of Epidemiology and
Biostatistics at the University of Western Ontario. His methodologic research
has contributed to the design and analysis of clinical trials, with publications
over the past ten years appearing in Biometrics, Statistics in Medicine and the
Journal of Clinical Epidemiology. Dr Donner served on the Editorial Board of
the American Journal of Epidemiology from 1991–1999, and is a Fellow of
the American Statistical Association.
Lelia Duley is an Obstetric Epidemiologist and MRC Senior Clinical
Research Fellow in the Resource Centre for Randomised Trials at the Institute
of Health Sciences, Oxford. Her areas of work interest include the design and
conduct of randomised trials, systematic reviews, perinatal epidemiology and
maternal mortality.
Alison Eastwood is currently a senior research fellow at the NHS Centre for
Reviews and Dissemination. Her background is in statistics and economics,
and throughout her career she has been involved in health services research.
Prior to joining the Centre over five years ago, she was a biostatistician on the
myocardial infarction Patient Outcomes Research Team at Harvard Medical
School in the US.
Sarah Edwards is Lecturer in Ethics in Medicine at the Centre for Ethics in
Medicine, Bristol. Her research interests major on ethics and medical
research, but include other methods and types of research besides Random-
ised Controlled Trials (RCTs). In particular, she is interested in the use
of patient records for epidemiologial studies that don’t ‘use’ the patients
personally or directly at all. In her previous post in the Department of Public
xiv The Advanced Handbook of Methods in Evidence Based Healthcare

Health and Epidemiology she completed a review of ethics and RCTs for
the NCCHTA.
Ray Fitzpatrick (editor) is Professor of Public Health and Primary Care,
Institute of Health Sciences, University of Oxford and Fellow and Dean,
Nuffield College, Oxford. He is currently chair of the MRC Health Services
and Public Health Research Board. His research interests focus on measure-
ment of outcomes of healthcare, such as health status, quality of life and
patient satisfaction and their use in clinical trials and evaluative research.
With Gary Albrecht he edited Quality of Life in Health Care (JAI Press,
Greenwich, Connecticut, 1994). With Stan Newman and other colleagues he
wrote Understanding Rheumatoid Arthritis (Routledge, London 1995). He co-
edited with Nick Black, John Brazier and Barnaby Reeves Health Service
Research Methods: A Guide to Best Practice (BMJ Books, London 1988).
With Gary Albrecht and Susan Scrimshaw he co-edited The Handbook of
Social Studies in Health and Medicine (Sage, London 2000).
John Gabbay qualified in medicine at Manchester in 1974. After seven years
at the University of Cambridge working on the social origins of medical
knowledge, he trained in public health. His early research included working
on the Templeton Study of NHS managers, and the development and critical
evaluation of audit methods. Since 1992, he has directed the Wessex Institute
of Health R&D, which now houses the National Co-ordinating Centre for
Health Technology Assessment, which runs the HTA programme for the
NHS. His recent research includes evaluations of several national pilot
schemes to implement clinical effectiveness in the NHS.
Heather Gage is a Senior Lecturer in Economics at the University of Surrey.
Her research interests focus on various aspects of health service delivery,
including the development of patient-centred outcome measures, international
comparisons of healthcare systems, and evaluations of new healthcare tech-
nologies. Recent publications have appeared in The European Journal of
Cancer Prevention, The Journal of Medical Screening, Archives of Neurology
and Health Technology Assessment.
Andrew Garratt is co-director of the National Centre for Health Outcomes
Development at the Institute of Health Sciences in Oxford. His research
interest is in patient-assessed health outcome measures.
Steve George is Senior Lecturer in Public Health Medicine at the University
of Southampton. In 1997 he formed and became the first Director of the
University of Southampton Health Care Research Unit (HCRU). He has
published widely in health services research, and his research grants total
over £4,000,000. He is an epidemiologist, and his interests include both the
application of epidemiological methods to the evaluation of medical care, and
the development and use of outcome measures. A particular interest is the
xv
Contributors

rising demand for healthcare, and the evaluation of attempts to meet it. In
1998 he published the SWOOP study, the first ever randomised controlled
trial of a nurse telephone consultation system in primary care, which formed
the evidence base for the new NHS Direct telephone service. He was also one
of the developers of the EORTC PAN-26, a questionnaire for measuring
outcomes in people with carcinoma of the pancreas and, with Professor Mike
Campbell, has published on sample size calculations and analysis of quality
of life data.
Simon Gilbody has first degrees in medicine and psychology and is a
member of the Royal College of Psychiatrists. He currently holds an MRC
Special Training Fellowship in Health Services Research at the University of
York and is an editor of the Cochrane Schizophrenia Group.
William J. Gillespie is Dean of the Dunedin School of Medicine of the
University of Otago in New Zealand. His recent work interests have been in
systematic reviews, and in clinical trials in orthopaedic surgery.
Paul P. Glasziou is a general practitioner and Associate Professor in Clinical
Epidemiology at the Department of Social and Preventive Medicine, Uni-
versity of Queensland. He is co-editor of the Journal of Evidence-Based
Medicine, a member of the editorial board of the British Medical Journal, and
co-chair of the Cochrane Collaboration Methods Group on Applicability and
Recommendations.
Adrian Grant is Director of the Health Services Research Unit, a 50-strong
Scottish Executive core funded research group within the University of
Aberdeen, UK. He leads the Unit’s programme on ‘Health Care Evaluation’.
His work has centred around the place and conduct of pragmatic trials in
health technology assessment, and he has a special interest in the evaluation
of non-drug technologies, particularly those, such as minimal access surgery,
whose evaluation is complicated by learning, skill and preference issues.
Alastair Gray is Reader in Health Economics at the University of Oxford
and Director of the Health Economics Research Centre, Institute of Health
Sciences, Oxford. He has previously held posts in health economics at
Aberdeen University, the Open University and the London School of Hygiene
and Tropical Medicine. His main research interests are the methodology and
application of economic evaluation in healthcare, priority setting methods, the
economics of ageing and the economics of medical negligence.
Colin Green is a Research Fellow in Health Economics at the School of
Health and Related Research at the University of Sheffield. His research
interests include the measurement and valuation of health outcomes, the
application of economic theory to NHS decision-making, and the societal
valuation of health and healthcare.
xvi The Advanced Handbook of Methods in Evidence Based Healthcare

Martin Gulliford is Senior Lecturer in Public Health Medicine at GKT
School of Medicine, King’s College London. His research interests are in
epidemiology as applied to health services research, and Caribbean health.
Sarah E. Hampson is Professor of Psychology and Health at the University
of Surrey, Guildford. Her research examines personality and beliefs as
determinants of health behaviours.
Gareth Harper is currently a Senior Research Officer in the Policing and
Reducing Crime Unit (PRCU), which is part of the Research, Development
and Statistics (RDS) Directorate of the Home Office. Prior to this, he was a
Research Health Economist at the University of Hertfordshire, where he
worked for Professor Joy Townsend in the Centre for Research in Primary
and Community Care (CRIPACC). His work was primarily on the methods of
Preliminary Economic Evaluation of Health Technologies. Prior to this,
Gareth had undertaken postgraduate research on modelling the costs of acute
care hospitals, and the efficiency measurement of hospitals.
Jo Hart is currently conducting doctoral research in health psychology at the
University of St Andrews. She has considerable experience working on
systematic literature reviews.
Emma Harvey graduated with a degree in Psychology in 1991, and spent
several years working in the NHS evaluating the provision of local services.
In 1995 she moved to the University of York, where she works in the field of
evidence based healthcare. She has developed interests in systematic review-
ing, multi-centre trials, the implementation of healthcare research findings
and effective professional practice. She is currently working on a national
trial of physical treatments for back pain, and has recently completed a PhD
on health professionals’ views and practice in relation to obesity.
Ian Harvey is Professor of Epidemiology and Public Health at the University
of East Anglia, Norwich. His research interests include pragmatic randomised
trials of non-pharmaceutical interventions in primary and secondary health-
care and social care; and the use of observational studies in the assessment of
health technologies.
David Henry is Professor of Clinical Pharmacology at the University of
Newcastle, Waratah, Australia. His interests span pharmaco-epidemiology,
pharmaco-economics, evidence based practice and health technology assess-
ment. He is a member of the Pharmaceutical Benefits Advisory Committee,
and the Health Advisory Committee of NHMRC, and chairs the Economics
Sub-Committee of PBAC. Invitations to make presentations on topics listed
above include US Congressional advisors, the shadow Ministry of Health
in the UK, the Cochrane Colloquium, The American Society of Clinical
Pharmacology and Therapeutics, International Society of Technology Assess-
ment in Health Care, the International Society of Pharmaco-Epidemiology,
and the International Society for Outcomes Research.
xvii
Contributors

Jenny Hewison is a Senior Lecturer in the School of Psychology at the
University of Leeds, specialising in the psychology of health and healthcare.
She has particular research interests in mother and child health and in
genetics, and more general methodological interests in the evaluation and
assessment of health technologies.
Suzanne Hill is Senior Lecturer in Clinical Pharmacology at the University
of Newcastle. Her research interests are in pharmaco-economics, the use of
evidence in public policy and decision-making, and in applying the results of
clinical trials in these settings as well as in clinical practice.
Jennifer Jackson is Senior Lecturer in Philosophy and Director of the Centre
for Business and Professional Ethics at the University of Leeds. She has
published in academic journals articles on abortion, euthanasia, competence,
surrogacy and consent. She has a book forthcoming on Truth, Trust and
Medicine (Routledge). She runs a postgraduate MA course on Health Care
Ethics.
Ann Jacoby is Professor of Medical Sociology in the Department of
Primary Care at the University of Liverpool. Prior to this she was a Reader in
Health Services Research at the Centre for Health Services Research at the
University of Newcastle. Her areas of work interest include assessment of
quality of life in chronic illness, particularly neurological (epilepsy, stroke);
patient based outcomes in the context of clinical trials; patient satisfaction and
research methods.
Alejandro Jadad is Chief at the Health Information Research Unit; Director,
McMaster Evidence-based Practice Center (designated by the US Agency for
Health Care Policy and Research); Co-Director, Canadian Cochrane Network
and Centre Investigator, Supportive Cancer Care Research Unit; Professor,
Departments of Clinical Epidemiology & Biostatistics, McMaster University,
Canada; and Associate Medical Director, Program in Evidence-based Care,
Cancer Care, Ontario. His current research interests focus on pain and
palliative care; the study of the information needs of different decision-
makers and their barriers to the use of best available research evidence; the
development of strategies to promote interaction and communication amongst
decision makers operating at different levels in the healthcare system; the role
of the Internet as a source of information for providers and consumers during
healthcare decisions; the design and execution of empirical methodological
studies to improve the design, conduct, reporting and dissemination of
individual studies and systematic reviews; the use of systematic reviews and
randomised trials as tools to gather new knowledge in healthcare; and the
development of strategies (with members of the public as co-investigators) to
transfer health research information to clinicians, patients, managers, policy-
makers, journalists and other lay decision-makers, and its use in conjunction
with their values and circumstances. He is Chair, Consumers and Health
xviii The Advanced Handbook of Methods in Evidence Based Healthcare

Informatics Working Group, American Medical Informatics Association;
Editor, Cochrane Consumers and Communication Review Group; Associate
Editor, Health Expectations, a new peer-reviewed journal that focuses on
patient participation in healthcare decisions. In 1997, Dr Jadad received the
‘National Health Research Scholars Award’, by Health Canada, to support his
program ‘Knowledge synthesis and transfer, consumers and evidence-based
health care’. In 1999, he received one of ‘Canada’s Top 40 Under 40’ awards
and one of the ‘Premier’s Research Excellence Awards’ in recognition for
the achievements and leadership in his multiple areas of interest, and his
contributions to society.
Katharine Johnston is a Research Fellow at the Health Economics Research
Centre, University of Oxford. She has previously held appointments at Brunel
University, the NHS Executive in Edinburgh and the University of Aberdeen.
Her research interests include methodological issues in economic evaluation;
the economics of screening (particularly breast screening); and the innovation
and diffusion of health technologies.
Alison Jones is based at the Thomas C. Chalmers Centre for Systematic
Reviews in Ontario. She co-ordinates a series of studies pertaining to the
effects of language publication of included randomised controlled trials on
the benefits of treatment interventions. She is also involved with the evalu-
ation of the CONSORT statement which was designed to improve the quality
of reports of randomised trials.
David Jones completed an MSc and PhD in mathematics before taking up a
series of posts in medical statistics and epidemiology in the NHS and several
universities. In 1991, he was appointed to the new Chair in Medical Statistics
at the University of Leicester. His research interests in the methodology of
health services research and clinical trials include meta-analysis, quality of
life assessment, and applications of Bayesian methods.
Steven Julious graduated in 1991 from the University of Reading with an
MSc in Biometry and took up a post as a medical statistician in the medical
school of the University of Southampton. Here he developed an interest in
the methodology of clinical trial design, in particular the aspect of sample
size estimation, and the statistical methodologies underpinning quality of
life assessment. In 1996, Steven left academia to take up a post at Glaxo-
Wellcome, and now works in the department of Clinical Pharmacology
Statistics, SmithKline Beecham.
Sandra Kiauka is currently working as a staff Epidemiologist at the Institute
for Health Care Systems Research in Germany. Previous to this she has
worked as a research fellow at the Medical Statistics Unit at the University of
Edinburgh and a clinical co-ordinator at Shared Care Informatics in Halifax,
Nova Scotia, Canada. Her areas of work interest include health service
xix
Contributors

research, healthcare reforms, service packages, public health, epidemiology
and information technology in public health research. She has a B.Sc. in
Health Information Sciences and an M.Sc. in Epidemiology.
Alan Kimber is Head of the Department of Mathematics and Statistics at the
University of Surrey. He has been a Fellow of the Royal Statistical Society
since 1979 and is a Chartered Statistician. His research interests include
survival analysis and reliability, statistical modelling and applications of
statistics to the medical and health sciences.
Terry Klassen is currently professor and chair of the Department of Pediatrics,
University of Alberta in Edmonton, Alberta, Canada. He is the co-ordinator
for the Child Health Field for the Cochrane Collaboration. His research
interests are in the use of clinical trials and systematic reviews for assessing
the effectiveness of interventions in child health.
Donna Lamping is Senior Lecturer and Head of the Health Services
Research Unit at the London School of Hygiene and Tropical Medicine. She
trained as a research psychologist (specialisation in health psychology,
psychometrics/measurement, personality) and worked at universities in
Canada (McMaster, McGill) and the USA (Harvard, Fordham) before coming
to LSHTM in January 1992. Her research is in two main areas: (i) methodo-
logical work on developing and validating patient-based measures of outcome
in several areas of medicine and surgery (e.g. prostatectomy, gynaecological
surgery, maternity services, HIV/AIDS, renal dialysis, back pain, venous
diseases of the leg, multiple sclerosis, ocular disease, neuro-rehabilitation,
cardiothoracic surgery, plastic surgery); and (ii) substantive work on quality
of life and psychosocial aspects of chronic illness (e.g. elderly people on
dialysis, HIV1 women, children at risk for hyperlipidaemia, CABG patients
randomised to conventional versus community care, dialysis patients being
treated in satellite units, arthritis patients).
James Lewsey is a Research Fellow in the Biostatistics Unit of the Eastman
Dental Institute, UCL. His current areas of interest include multilevel
modelling and Bayesian analysis in oral health services research, meta-
analysis, and generalised linear modelling.
Alastair Leyland is a Research Scientist in the MRC Social and Public
Health Sciences Unit at the University of Glasgow. He is head of a pro-
gramme concerned with measuring health, variations in health and the
determinants of health in Scotland, and his current areas of interest include
applications of multilevel modelling to health data and the spatial modelling
of disease.
Richard Lilford (editor) was born in Cape Town in 1950, received his
schooling in Johannesburg, and became a doctor in that city. He moved to
xx The Advanced Handbook of Methods in Evidence Based Healthcare

Cape Town where he specialised in obstetrics and gynaecology at the Groote
Schuur Hospital, before coming to work in London, first as a registrar and
then a senior registrar. He was appointed as a consultant at Queen Charlotte
Hospital in 1982, and was then made a professor in Leeds in 1984, where he
ran a department for seven years. He then headed the Institute of Epidemiology
and Health Service Research at the University of Leeds for a further four
years before coming to his present job where he directs the government’s
programme of research in the West Midlands, directs the central programme
of methodological research, and advises the government on trials. He is also
a part-time Professor in the Department of Public Health and Epidemiology at
the University of Birmingham.
Julia Lowe completed her medical training in the UK and then moved to
Australia. She worked part-time in private practice while her children were
small, completing a masters degree in clinical epidemiology in her spare time.
She moved back to the public sector in 1991 and became Director of General
Medicine at John Hunter teaching hospital in 1996. She has established
research programmes in diabetes and cardiac failure as well as maintaining an
interest in health services research. She is criticism editor for the Cochrane
Heart Group.
David Machin is currently Professor of Clinical Trials Research at the
University of Sheffield, former Director of the National Medical Research
Council, Clinical Trials and Epidemiology Research Unit, Singapore (1996–9)
and Chief Medical Statistician MRC Cancer Trials Office, Cambridge
(1988–98). His degrees include a MSc from the University of Newcastle upon
Tyne, and PhD from the University of Southampton. He is an honorary
Member of the Royal College of Radiologists. He has published more than
200 articles and several books on a wide variety of topics in statistics and
medicine. His earlier experience included teaching at the Universities of
Wales, Leeds, Stirling and Southampton, a period with the European Organisa-
tion for Research and Treatment of Cancer, Brussels, Belgium and at the World
Health Organization in Geneva, Switzerland working on contraceptive trials.
He is an editor of Statistics in Medicine and member of the Editorial Board of
the British Journal of Cancer.
Rachel MacLehose is currently working as a Public Health Epidemiologist at
South Essex Health Authority. Previous posts include working as an environ-
mental epidemiologist at the Chemical Incident Response Service at Guy’s
and St. Thomas’ Hospital Trust and as research associate at the University of
Bristol. Rachel holds an MSc in Environmental Epidemiology and Policy
from the London School of Hygiene and Tropical Medicine.
Theresa Marteau is Professor of Health Psychology and Director of the
Psychology and Genetics Research Group at King’s College, London. Over
the past 15 years she has been conducting research on psychological aspects
xxi
Contributors

of health risk assessment. The work has covered genetic testing in pregnancy,
adulthood and childhood, as well as population based screening programmes.
The conditions studied include heart disease, cervical, breast and bowel
cancer and cystic fibrosis. The aim of this research is to understand responses
as a first step towards evaluating different methods of communicating
information to promote understanding, reduce emotional distress and enhance
health-promoting behaviours. She has published over 100 peer-reviewed
articles in this and related areas and is co-editor of The Troubled Helix (1996;
paperback edition: 1999), a book reviewing the psychological implications of
the new human genetics.
Elaine McColl has a first degree in Geography and Economics and an MSc in
Applied Statistics. She is a native of Dublin and worked for several years in
the Department of Statistics, Trinity College, Dublin, where she taught formal
courses and provided informal advice on survey design and analysis. Since
1987, she has been a member of the Centre for Health Services Research,
University of Newcastle upon Tyne, where she currently holds the post of
Senior Research Associate. Her main research interests are in the assessment
of health-related quality of life and in survey methodology. She has recently
been awarded an NHS Primary Care Career Scientist award, to develop a
programme of research into quality of life in chronic disease. Elaine was
principal investigator for a recently completed literature review Designing
and using patient and staff questionnaires which was funded by the NHS
Health Technology Assessment Programme and upon which the findings
reported here are based.
Lorna McKee is Director of Research at the Department of Management
Studies, University of Aberdeen and was previously principal researcher at
the Centre for Corporate Strategy and Change, University of Warwick. She is
joint author of Shaping Strategic Change (1992) with Andrew Pettigrew and
Ewan Ferlie, and has published in an extensive range of academic and
professional journals. She has recently investigated medical leadership,
primary care management and biomedical innovation transfer processes in
research projects funded by the ESRC, CSO and NHS. She regularly acts as
a consultant to healthcare organisations across the United Kingdom, and
jointly co-ordinates the Multi-Disciplinary Primary Care Leadership Pro-
gramme leading the module on Developing and Leading Effective Teams.
Martin McKee is Professor of European Public Health at the London School
of Hygiene and Tropical Medicine and co-director of the School’s European
Centre on Health of Societies in Transition. His main research interest is
health and healthcare in central and eastern Europe and the former Soviet
Union, but he also has a more general interest in the interpretation and use of
evidence in policy-making.
Klim McPherson is Professor of Public Health Epidemiology in the Cancer
and Public Health Unit at the London School of Hygiene and Tropical
xxii The Advanced Handbook of Methods in Evidence Based Healthcare

Medicine. He has written over 300 refereed papers during his 30-year career
in epidemiology. His particular interests include the relationship between
breast cancer and hormone supplements such as HRT and oral contraceptives,
the incidence of coronary heart disease, the effects of tobacco and alcohol and
the widening of public health training to include those who do not come
from a conventional and traditional medical background. He is a member of
the editorial board of various refereed journals, including the Journal of
Epidemiology and Community Health, and his membership of different
organisations includes the Board of the Faculty of Public Health Medicine
and the Council of the UK Public Health Association. He has been the author
and co-author of many books and book chapters such as Hormones and
Cancer, ed. by O’Brien and MacLean, published by the RCOG Press
(Chapter 16), a chapter in Health issues related to consumption, ed. by Ian
MacDonald, published by Blackwell Science, and with Prakash Shetty, he
edited Diet Nutrition and Chronic Disease, Lessons from Contrasting Worlds,
published by John Wiley & Sons.
David Moher is an Assistant Professor in the Departments of Pediatrics,
Epidemiology and Community Medicine and Medicine, Faculty of Medicine,
University of Ottawa, and Director, Thomas C. Chalmers Centre for System-
atic Reviews at the Children’s Hospital of Eastern Ontario Research Institute.
During the past ten years his interests have focused on how randomised trials
and meta-analyses of these studies are conducted and reported. He has been
particularly interested in how biases individually and collectively influence
estimates of treatment efficacy.
Michael Moher is currently a GP and Research Fellow at the Department of
Primary Health Care at the University of Oxford. He is currently involved as
medical co-ordinator, in research on the evidence based secondary prevention
of coronary heart disease in primary care. This research is comparing, in a
randomised controlled trial, three methods of promoting secondary prevention
of heart disease in primary care: audit and feedback, the introduction of
structure of records, registers and recall, and the introduction of nurse-run
clinics.
Graham Mowatt has, since April 1997, been Review Group Co-ordinator for
the Cochrane Effective Practice and Organisation of Care Group (EPOC).
The focus of EPOC is on producing systematic reviews of interventions
designed to improve health professional practice and the delivery of effective
health services. This includes various forms of continuing education, quality
assurance, informatics, financial, organisational and regulatory interventions
that can affect the ability of health professionals to deliver services more
effectively and efficiently. Prior to working on the NHS Health Technology
Assessment Programme project on assessing fast-changing technologies he
was employed by Grampian Health Board for four years on a health service
project which was concerned with primary care technological innovation. He
also possesses a Master of Business Administration (MBA) degree.
xxiii
Contributors

Elizabeth Murphy is a senior lecturer in Sociology and Social Policy at the
University of Nottingham. Before moving to Nottingham, she was a research
fellow in the Department of Primary Medical Care at the University of
Southampton. Her major research interests lie in the medical sociology and
the sociology of food. Her doctoral work was concerned with the lay health
concepts held by people with non-insulin-dependent diabetes and their
implications for their response to medical advice about lifestyle. This interest
in responses to behavioural advice has been extended in more recent studies
of patients with other chronic disorders and, most recently, a study of the
decisions mothers make in relation to infant feeding. While she has used both
qualitative and quantitative methods in her research, she has a particular
interest in the rigorous application of qualitative methods to policy and
practice-related issues.
Maggie Murphy is a lecturer in Social Psychology in the School of
Humanities and Social Sciences, University of Glamorgan. Her research
interests include social psychological aspects of healthcare provision and
social psychological determinants of health behaviours.
Gordon Murray is Professor of Medical Statistics at the University of
Edinburgh, having previously worked at the University of Glasgow where he
was Director of the Robertson Centre for Biostatistics. A mathematician by
training, Professor Murray was recently elected to Fellowship of the Royal
College of Physicians of Edinburgh. His main interest is in multicentre
clinical trials, particularly in the cardiovascular and cerebrovascular areas. He
has also worked extensively in prognostic modelling, and in the application of
such methodology to permit casemix adjustment in clinical audit.
Jonathan Myles is a statistician at the MRC Biostatistics Unit. His interests
include the application of Bayesian statistics to medicine including the use
of Markov Chain Monte Carlo for complex models in epidemiology and
population genetics, the assessment of breast cancer screening effectiveness,
and problems in AIDS epidemiology.
Dianne O’Connell is a biostatistician with a strong interest in evidence based
medicine. She was a Senior Brawn Research Fellow for two years (1997–99),
and is now a Senior Research Academic in the Discipline of Clinical
Pharmacology, Faculty of Medicine and Health Sciences, University of
Newcastle, Australia. Her research interests are in methods for conducting
systematic reviews and methods for translating the results of medical research
(in particular randomised controlled trials and systematic reviews) into
information that can readily be used in clinical decision-making and policy
formulation. This work includes developing methods for identifying in which
individuals an intervention is more likely to do good than harm (the topic of
Chapter 4) and the effects of information framing (how the data about an
intervention’s effectiveness are presented) on clinical decision-making.
xxiv The Advanced Handbook of Methods in Evidence Based Healthcare

Ba’ Pham is a biostatistician with the Thomas C. Chalmers Centre for
Systematic Reviews and an associate member of the Children’s Hospital of
Eastern Ontario Research Institute. His main research interest is to perform
empirical studies evaluating statistical methods used to estimate the risks and
benefits of healthcare interventions.
Robin Prescott is Director of the Medical Statistics Unit at the University of
Edinburgh and has research interests in clinical trials stretching back for 30
years. He has particular interests in the application of mixed models in
medicine and in the methodology associated with cross-over trials.
James Raftery is Director of Health Economics Facility and Professor of
Health Economics at the Health Services Management Centre (HSMC),
University of Birmingham. He joined HSMC to become director of the
Health Economics Facility in 1996. Before moving to HSMC, James worked
at the Wessex Institute for Research and Development and for Wandsworth
Health Authority. He has also worked on secondment as an economic adviser
to the Department of Health and to the National Casemix Office. James’
current projects include: involvement in several health technology evaluations
(anti-coagulation clinics, chondrocyte implantation, stroke services); analysis
of the public expenditure impact of new health technologies; the use of
routine data in health technology assessment (a national R&D project); and
methods of costing diseases, interventions and programmes. He has also
edited a series on economic evaluation for the British Medical Journal. Other
recent publications include the three volume series: Health care needs
assessment: the epidemiologically based needs assessment reviews. First and
second series, which were edited by Andrew Stevens and James Raftery.
Barnaby Reeves is Senior Lecturer in Epidemiology and Director of the
Clinical Effectiveness Unit at the Royal College of Surgeons. Following a
doctorate in Experimental Psychology, he worked on the development and
evaluation of psychophysical tests in ophthalmology for several years. His
interest in healthcare evaluation led him to study epidemiology, following
which he was appointed as a senior lecturer in health services research at the
University of Bristol. In this post, he designed and set up a wide range of
evaluations, including surgical procedures, physiotherapy, nursing, rehabilita-
tion and educational interventions. His main interests are: use of non-
randomised or hybrid designs for evaluating interventions, especially surgery,
the measurement of the quality of healthcare provision and evaluations of
diagnostic accuracy.
Glenn Robert is currently Research Fellow at the Health Services Manage-
ment Centre, University of Birmingham. Previous to this he was a Research
Fellow in the Health Economics Research Group at Brunel University and a
researcher at the National Co-ordinating Centre for Health Technology
Assessment at the University of Southampton. His recently submitted PhD
xxv
Contributors

thesis examines the use of different sources for identifying new healthcare
technologies prior to their widespread adoption by the NHS, and makes
recommendations regarding the establishment and operation of an early
warning system in the United Kingdom.
Paul Roderick trained in general medicine and then public health medicine.
He has been senior lecturer in Public Health Medicine at Southampton
University since 1993. His main research interests are in evaluating the equity
and effectiveness of healthcare delivery, and in defining the epidemiology of
chronic renal failure. He is a member of the Steering Committee of the UK
Renal Registry.
Sue Ross co-ordinated/edited Chapter 3, ‘Factors That Limit the Number,
Progress and Quality of Randomized Controlled Trials: a Systematic Review’
on behalf of a large group of authors. Sue has worked as a health services
researcher for 12 years. She is currently Co-ordinator of Clinical Research at
Mount Sinai Hospital, Toronto. At the time the review was carried out, she
was Trials Programme Co-ordinator in the Health Services Research Unit at
the University of Aberdeen, managing a programme of health technology
evaluation trials, mainly in surgery and orthopaedics. Prior to that she had
been involved in surgical and primary care trials.
Daphne Russell is a biostatistician based in North Yorkshire. She was
educated at the Universities of Cambridge (graduating in Mathematics and
Mathematical Statistics) and Aberdeen (Public Health). She has held aca-
demic appointments in the Universities of Essex, Newcastle upon Tyne,
Aberdeen and Hull (as Director of the University Statistical Support Unit).
Ian Russell has been Founding Professor of Health Sciences at the University
of York since 1995. He was educated at the Universities of Cambridge
(graduating in Mathematics), Birmingham (Statistics) and Essex (Health
Services Research). He has held academic appointments in the Universities of
Newcastle upon Tyne, North Carolina, Aberdeen (as Director of the Scottish
Health Services Research Unit) and Wales (while Director of R&D for NHS
Wales).
Colin Sanderson is Reader is Health Services Research (HSR) at the London
School of Hygiene and Tropical Medicine. He has a background in opera-
tional research and epidemiology, and has been involved in HSR for over 15
years. His research interests include monitoring quality of care, quantitative
modelling for policy evaluation, equitable allocation of healthcare resources
and needs assessment.
Jasminka Sarunac obtained a Bachelor of Pharmacy at the University of
Belgrade. He went on to be a pharmacist and subsequently Director of the
Pharmacy Department at Clinical Centre Dr Lj. D. Kraljevo in Yugoslavia.
He has completed a postgraduate course of Clinical Pharmacy Practice and
xxvi The Advanced Handbook of Methods in Evidence Based Healthcare

has a Master of Medical Science in Pharmaco-epidemiology Studies from the
University of Newcastle, NSW, Australia. He has published work on “GPs’
beliefs regarding benefits and risks of Hormone Replacement Therapy: Does
framing of information affect enthusiasm for prescribing HRT”. He is
currently employed by the Pharmacy Department at John Hunter Hospital,
Newcastle, Australia in the capacity of Hospital Pharmacist. His areas of
research interest include prescribing practices and evidence based medicine.
Trevor Sheldon is Professor and Head of the Department of Health Studies at
the University of York and co-director of the York Health Policy Group. He
trained in medicine, economics and medical statistics. His main research
interests are in resource allocation, healthcare quality, the organisation and
delivery of healthcare, and the evaluation of healthcare interventions. He was
director of the NHS Centre for Reviews and Dissemination at the University of
York from 1993–1998 and manager of the Effective Health Care bulletins.
Sue Shepherd is currently a Specialist Registrar in Public Health Medicine,
and her special interests are Communicable Disease and Health Needs
Assessments. Prior to this she was a Clinical Lecturer in Epidemiology. She
has been involved with large epidemiological studies and is presently a co-
grant holder in a randomised controlled trial looking at radiotherapy in older
women with early breast cancer.
Fujian Song has an MD in public health medicine (China) and a PhD in
health services studies (Leeds). He is a senior research fellow at the NHS
Centre for Reviews and Dissemination at the University of York. Current
areas of work interests include methodology of systematic reviews and
evaluation of healthcare interventions.
Jennifer Soutter has a first degree in English and Psychology; her PhD is
titled ‘Archetypal Elements in the Poetry of Sylvia Plath’. Returning to work
after completing six years of higher education, she has worked in health
services research for the past ten years, over six of these years being at the
Centre for Health Services Research at the University of Newcastle upon
Tyne. Her current post is as research associate at the Department of Child
Health at the Royal Victoria Infirmary working on a project with children
diagnosed with Duchenne muscular dystrophy. This links with work carried
out with children with life-limiting illnesses some six years ago.
David Spiegelhalter is a Senior Scientist at the MRC Biostatistics Unit in
Cambridge. His current interests include the development and dissemination
of Bayesian methods in biostatistics, including applications in health technol-
ogy assessment and the analysis of performance indicators. He has published
widely on Bayesian theory and applications, and has been instrumental in the
development of the BUGS and WinBUGS software for Bayesian analysis
using graphical models and Markov Chain Monte Carlo methods.
xxvii
Contributors

Nick Steen is a medical statistician based at the Centre for Health Services
Research at the University of Newcastle upon Tyne with special interests in
the design and analysis of cluster randomised trials and the development of
health outcome measures. He is currently involved with a number of trials
involving the assessment of health technology.
Jonathan Sterne is a Senior Lecturer in Medical Statistics in the Department
of Social Medicine at the University of Bristol. His research interests include
methods to detect bias in meta-analysis and systematic reviews, the use of
statistical methods for the analysis of clustered and longitudinal data, and the
epidemiology of asthma and allergic diseases.
Andrew Stevens (editor) is Professor of Public Health and Head of Depart-
ment in the Department of Public Health and Epidemiology at the University
of Birmingham, England. His interests concern health technology assessment,
evidence based healthcare, and needs assessment. He is (co-)director of the
National Horizon Scanning Centre, providing advance notice of new medical
innovations to government; and a founder member of Euro-Scan, a collabora-
tive healthcare horizon scanning group across Europe and Canada. He edits a
series of publications on Health Care Needs Assessment and has established
a Health Care Evaluation Service which reviews, models cost–utility and
appraises the evidence on new and existing health technologies. He was
formerly the first Director of the National Co-ordinating Centre for Health
Technology Assessment at Southampton University.
Alex Sutton is a Lecturer in Medical Statistics at the University of Leicester.
His research interests encompass several aspects of systematic review meth-
odology, including the use of Bayesian methods; methods for addressing
publication bias; and methods for the synthesis of results from studies with
different designs. He is an active member of the Cochrane Collaboration,
contributing to both the statistical and non-randomised studies methods
groups.
Hilary Thomas is a Lecturer in the Department of Sociology, University of
Surrey. She is President of the European Society of Health and Medical
Sociology (1999–2003). Her research interests include the sociology of health
and illness, reproduction and women’s health, and the sociology of time.
Lois Thomas is a Senior Research Associate at the Centre for Health
Services Research and the Department of Epidemiology and Public Health at
the University of Newcastle upon Tyne. A nurse by background, her research
interests are in stroke, user satisfaction and evidence based practice.
Roger Thomas is a Research Director in the Survey Methods Centre of the
National Centre for Social Research (formerly SCPR) and also Director of the
ESRC-funded Centre for Applied Social Surveys (CASS). He has more than
xxviii The Advanced Handbook of Methods in Evidence Based Healthcare

30 years’ experience as a quantitative social survey project designer and
manager in many different topic areas. Much of this was in Social Survey and
Census Divisions of the Office for National Statistics, where he was assistant
director responsible for survey methodology and also for the design and
conduct of major government suveys. In the Survey Methods Centre he acts
as consultant on survey design problems, works on methodological research
projects and writes on survey methods. As Director of CASS he teaches
survey methods courses at postgraduate level and heads a team which develops
and disseminates a Social Survey Question Bank, containing examples and
commentary on questionnaire design, on the World Wide Web.
Jim Thornton is Reader in Obstetrics and Gynaecology at Leeds University,
and honorary consultant at Leeds General Infirmary. He qualified in medicine
in Leeds in 1977, and worked for four years in Africa before training as a
specialist. He has organised and participated in trials of many pregnancy
interventions, and is currently principal investigator of the MRC Growth
Restriction Intervention Trial (GRIT). Since 1997 he has been Chairman of
Northern and Yorkshire Multicentre Research Ethics Committee. He is an
advocate of free-market solutions to healthcare problems, and was in the
news in 1999 for suggesting that much NHS health screening would be better
done privately!
Joy Townsend is a health economist and statistician, director of the Centre
for Research in Primary and Community Care and Professor of Primary
Healthcare. For many years she was senior scientist with the Medical
Research Council. Her major interests are in prioritisation of research using
ex ante modelling of health benefits and costs, randomised controlled trials of
new health services and economics of tobacco control.
Peter Tugwell joined the University of Ottawa as Chairman of the Depart-
ment of Medicine and as Physician-in-Chief at the Ottawa General Hospital
(now the Ottawa Hospital) in July 1991. He is the author/co-author of over
170 published articles, which focus principally on technology assessment and
the clinical epidemiology of the rheumatic diseases. He is actively involved in
several international organisations. He is the editor of the Musculoskeletal
review group within the International Cochrane Collaboration Centre. His
research interests are in the areas of clinical epidemiology, quality of life,
economic evaluation, and technology assessment both within the context of
industrialised countries as well as developing countries.
Obioha C. Ukoumunne obtained an MSc in Social Research Methods and
Statistics from City University in 1994. He has worked as a Research
Statistician at the Department of Medical Statistics and Evaluation, within
Imperial College School of Medicine since 1997. His chief research interest is
in methods appropriate for the design and analysis of cluster randomised
trials. His experience of applied medical statistics is mainly in the fields of
xxix
Contributors

psychiatry and general practice. His previous research posts were at the
Department of General Practice and the Department of Public Health Sciences,
both within GKT School of Medicine. Work on Chapter 17 was completed
whilst based at the Department of Public Health Sciences.
Marjon van der Pol is an economist who works for the Health Economics
Research Unit at the University of Aberdeen.
Sarah Walker is currently a Research Officer at the Office for National
Statistics where she is working with others on improving the coverage of the
next UK Population Census. She was a Research Assistant in the Medical
Statistics and Computing Department at the University of Southampton and
undertook a years’ research into ‘Sample size determination for quality of life
measures’. She investigated QoL measures commonly used with cancer
patients which generated categorical responses. She also worked on a variety
of clinical trials and survival analyses. Her research interests are still in
medical analysis, particularly in the field of disability and housing which, at
present, she undertakes on a consultancy or voluntary basis.
xxx The Advanced Handbook of Methods in Evidence Based Healthcare

I N T R O D U C T I O N
Methods in Evidence Based Healthcare
and Health Technology Assessment:
An Overview
A N D R E W S T E V E N S , R A Y F I T Z P A T R I C K ,
K E I T H A B R A M S , J O H N B R A Z I E R
and R I C H A R D L I L F O R D
EVIDENCE BASED MEDICINE, EVIDENCE
BASED HEALTHCARE AND HEALTH
TECHNOLOGY ASSESSMENT
Evidence based medicine, evidence based health-
care and health technology assessment are terms
representing a number of important recent devel-
opments in healthcare. The phrase ‘health tech-
nology assessment’ was coined by the United
States Office of Technology Assessment in 1972
as part of a wide-ranging interest on the part of
Congress in the evaluation of new technologies
across not just healthcare, but all major sectors
of importance to the US economy.1
Elsewhere,
the term was picked up a decade or so later, such
that by 1985, there was an International Society
for Health Technology Assessment, an inter-
national organisation aiming to promote scien-
tifically based assessment of technologies in
healthcare, including drugs, devices, medical
and surgical procedures, as well as organisa-
tional and administrative systems. The UK
adopted the term ‘health technology assessment’
with the funding in 1993 of a specific pro-
gramme under that name as part of its National
Health Service Research and Development
strategy.2
‘Evidence based medicine’ has its roots in clin-
ical epidemiology, owing much to Cochrane’s
(1972) critique of the effectiveness and effi-
ciency of health services.3
Cochrane argued that
many commonly used treatments and investiga-
tions in contemporary healthcare systems have
not been shown to be effective in terms of clear
and convincing evidence. A growing literature
on geographical variations, including Wenn-
berg’s notable studies in New England, has
underlined how diverse are health professionals’
ways of managing similar health problems and
how great is the deficit of, and the gap between
scientific evidence and clinical practice.4
Sackett (1996) describes evidence based
medicine as ‘the conscientious, explicit, and
judicious use of current best evidence in making
decisions about the care of individual patients’.5
The term ‘evidence based healthcare’ extends
the scope to those making decisions about not
just patients but also populations. By widening
the audience, it has also widened the scope of the
subject matter to include not only the evidence for
the effectiveness of elements of healthcare, but
also its costs and other ramifications. Further-
more, it is acknowledged in both health technol-
ogy assessment and evidence based healthcare,
that the health ‘technologies’ of interest are very
wide, concerning not only pharmaceuticals and
healthcare devices, but also procedures, settings,
healthcare personnel and more diffuse innova-
tions including, for example, managerial, IT and
policy changes in general. In short, evidence
based healthcare and health technology assess-
ment have converged to cover an interest in the
costs and benefits of all activities within the

healthcare system, and indeed even those for
which responsibility lies outside the boundaries
of the healthcare service.
THE HEALTH TECHNOLOGY ASSESSMENT
REVOLUTION
The growth of health technology assessment and
evidence based healthcare has lately become
something of a revolution. There has been an
explosion of journals, conferences, books and
research expenditure on health technology
assessment. In almost every OECD country, it
has become clear that the health services cannot
endlessly sustain an acceptance of all innova-
tions, whether they have been evaluated or not,
and whether they are affordable or not. Health
Technology Assessment (HTA) has become an
important part of a strategy to contain costs and
maximise value from available healthcare
resources. This has led to some countries devel-
oping guidelines for the reporting of economic
evaluations of pharmaceuticals and other tech-
nologies, such as in Canada,6
Australia7
and
the UK.8
A single new pharmaceutical, beta-
interferon – for multiple sclerosis – has the
capacity to cost the UK National Health Service
the GDP of many small countries, and cost the
United States Health Services the GDP of a
number of large ones. A similar story is true of
an increasing armoury of new pharmaceuticals.
The effectiveness of not just new technolo-
gies, but also of many established ones, has been
widely questioned. The problem has been one of
both ineffective healthcare being promulgated
into health services, and of therapeutically
advantageous techniques long being ignored.
Antman et al. (1992) demonstrate examples of
both, with huge time lags between the arrival of
conclusive evidence against antidysrrhythmics,
and supportive of thrombolytics in myocardial
infarction, and the recognition of these facts in
medical text books, quite apart from clinical
practice.9
So there has been a sea change in thinking
about health care, both clinically and at a policy
level. There has been a proliferation of attempts
to satisfy the demand for evidence based health-
care, covering the growth and scope of interest
of funding bodies, the development of network-
ing and co-ordinating bodies, the development
of multiple new publications and dissemination
media (from new journals to electronic data-
bases to compendia of electronic databases), and
systems for developing skills in interpreting
evidence. Most importantly, there have been a
number of innovations in the development of the
research base.
FEATURES OF THE RESEARCH BASE
It is now widely recognised that research for
health technology assessment includes not just
primary research (data gathering), but also sec-
ondary research (a systematic review of primary
research evidence). Indeed, as the latter becomes
more formalised, disciplined and by conse-
quence narrow, tertiary research synthesising
and making accessible multiple primary and
secondary research products is also developing.
The critical features of good primary research
are being increasingly recognised as indispens-
able: appropriately posed questions, studies
designed to reduce bias, and with patient-
relevant outcomes, conducted without bias,
ethically, and with suitable quality control,
drawing conclusions which flow from results,
and published regardless of the outcome.
In the same way the critical features of sec-
ondary and tertiary research in HTA are agreed:
systematic and reproducible reviews using clear
search strategies, formal critical appraisal of
contributory studies, grading of the evidence
from these studies according to their quality and
size, selection of relevant outcomes from pri-
mary research, and valid methods for integrating
the evidence.10
THE METHODOLOGY OF HEALTH
TECHNOLOGY ASSESSMENT
The growth of demand for health technology
assessment does not of itself guarantee the
appropriateness and validity of the material it
generates. An interest in methods employed to
ensure the quality of evidence has emerged. This
is manifest, for example, in the number of
different professions associated with HTA,
including epidemiologists, health economists,
sociologists, statisticians, clinicians and others.
Each is in a position to supplement – and to
some extent confront – the methods used by the
others. And the fertile mix of ideas is becoming
increasingly apparent. A particular manifesta-
tion of the interest in the methods of HTA was
the United Kingdom’s decision specifically to
have a programme of research on the method-
ology of HTA. It posed a series of questions
which fit into the following broad categories:
1 Methods in clinical trials,
2 The place of observational and qualitative
research,
3 The measurement of benefits and costs,
4 Analytical (statistical) methods which quan-
tify the uncertainty of the findings,
2 The Advanced Handbook of Methods in Evidence Based Healthcare

5 Methods of reviewing, synthesising, and
modelling findings, and
6 Methods for identifying gaps in the evidence
and selecting priorities.
The chapters of this book are arranged in this
logical way. By bringing together the current
state of art of knowledge on the methods for
health technology assessment, they offer answers
to a wide range of methodological questions
fundamental to HTA as follows:
How do we identify potential areas of con-
cern? (Chapter 25, Horizon Scanning)
f
Which of these topics is most urgent?
(Chapter 27, Preliminary evaluation)
f
When does the question need to be tackled?
(Chapter 26, Timing)
Having chosen the technologies and questions of
interest, and before gathering data afresh, we
must ask how much data/evidence is there
already?
Can we be sure we have all relevant evi-
dence? (Chapter 21, Publication bias)
f
Having gathered the existing evidence, how
can we synthesise it? (Chapter 22, Meta-
analysis)
f
Can we rely on the syntheses? (Chapter 23,
Assessing the quality of randomised con-
trolled trials in meta-analysis)
If the existing data are unsuitable or inadequate,
it follows that new data need to be collected.
What measurements are most useful to
make? (Chapter 11, Patient-based outcome
measures)
f
How should questionnaires be used to gen-
erate these data? (Chapter 15, Question-
naires)
f
Can these data be combined to generate
comparable measures of health status?
(Chapter 12, Health Status Measures)
f
And how can we measure costs as well as
outcomes? (Chapter 13, Collecting resource
data)
f
How do these costs and benefits become of
less immediate value as they are further in
the future? (Chapter 14, Discounting)
f
And how can we ensure that we have made
the best of the cost data? (Chapter 18,
Handling uncertainty in cost data)
f
How should new data be collected –
through new randomised controlled trials,
or by collecting data on observed activity?
(Chapters 6 and 7, Observational methods)
f
If observational methods are deemed
adequate, will routine data do? (Chapters 8
and 9, Routine data)
f
Or might qualitative methods suffice in some
circumstances? (Chapter 10, Qualitative
methods)
f
Or not using new data at all, but making the
best of consensus in some instances? (Chap-
ter 24, Consensus methods)
f
If new trials are decided upon – how gen-
eralisable are they? (Chapter 4, General-
isability)
f
What are the limiting factors in trial design?
(Chapter 3, Limiting factors in trial design)
f
3
Introduction

For example, are they limited by the ethics
of undertaking trials? (Chapters 1 and 2)
f
Should general account be taken of the
placebo effect? (Chapter 5, Placebo effect)
f
Whatever the methods used for gathering
data, how can we best analyse and interpret
it? (Chapter 16, Bayesian methods)
f
When data have been collected on both
quality and length of life how should they
be analysed? (Chapter 20, Quality of life
and survival analysis)
f
How large should the sample sizes be of the
newly collected data? (Chapter 19, Sample
size issues)
f
Are there special study designs required
when the population receiving an inter-
vention is clustered into schools, practices
or other units (Chapter 17, Area-wide
interventions)
It can be seen that this sequence, in order to
follow a logic of asking what the question is and
whether it has yet been answered, disrupts the
contents order of the book. In the main volume
we have stayed with the familiar logic of trials
first, horizons last, but either order would be
sustainable.
METHODS FOR UNDERTAKING
METHODOLOGICAL RESEARCH
All of the contributions in this volume have had
to select methods to generate their suggestions
for methodology in HTA. Principally these have
been those of synthesising the evidence from
other pieces of work. This can take the form of
either synthesising primary research (e.g. a
review of studies seeking to compare consent
rates for clinical trials by method of invitation)
or synthesising arguments (e.g. arguments over
whether a fixed or random effects statistical
model is more appropriate for meta-analysis). In
some cases authors have also sought to collect
new data themselves (e.g. Leyland’s use of
routine data), or indeed have triangulated their
evidence by drawing on a variety of different
methods (e.g. Roberts getting clues on how to
horizon scan).
CONCLUSION
The contributions to this volume, and indeed the
editorial collaboration in collating the volume,
have distinctive origins that are worth under-
lining. The UK National Health Service estab-
lished a national Research and Development
(NHS R&D) initiative of which the Health
Technology Assessment (HTA) Programme has
been a major component.2
It is hoped that sev-
eral features make this an important develop-
ment in international, as well as national, terms.
In the first place, an elaborate process was
established for inviting ideas for research from
the widest range of health professionals, re-
searchers, consumer interest groups and others.
Secondly, expert panels were asked carefully to
develop and prioritise research ideas submitted.
Thirdly, research that has been prioritised for
funding is commissioned by advertising to
researchers throughout the world to submit ten-
ders, with extensive peer review to determine
research groups to take forward projects. Great
emphasis has been placed on the need for inter-
disciplinary strengths for research groups to be
commissioned to undertake work for the HTA
programme. The methodology programme of
research has been a distinctive strand of the
HTA programme as a whole. Because of the
extensive process of consultation to identify
methodological questions and of selecting invest-
igators, and because of the thorough peer-
reviewing of outputs, we are confident that the
contributions reported here can be considered an
authoritative summary of current methodo-
logical issues in HTA. Every chapter is a sum-
mary of a more extensive study that has gone
through many steps of evaluation before emerg-
ing to form a monograph for the NHS R&D
programme. Efforts were also made to achieve
coherence to this body of methodological work
by workshops and other forms of communica-
tion to share experiences and results.
This volume has not set out to provide readers
with a basic introduction to the elementary
aspects of topics such as the design of random-
ised controlled trials or cost–benefit analysis
that can be found elsewhere. It does, however,
attempt to tackle the frontiers of health tech-
nology assessment. The book brings together a
unique collection of contributions that will guide
the reader through current thinking and debates

about the state and direction of research methods
in health technology assessment. If the volume
stimulates further debate as how to improve our
methods of health services research, it will have
achieved its goal.
REFERENCES
1. US Congress Office of Technology Assessment.
Identifying Health Technologies That Work:
Searching For the Evidence. Washington: US
Government Printing Office, 1994.
2. Department of Health NHS Executive. Research
and Development In the New NHS: Functions
and Responsibilities. London: Department of
Health, 1994.
3. Cochrane A. Effectiveness and Efficiency Random
Reflections on the Health Service, Leeds: Nuffield
Provincial Hospitals Trust, 1972.
4. Wennberg JE, Freeman JL, Culp WJ. Are hospital
services rationed in New Haven or over-utilised
in Boston? Lancet 1987; i: 1185–9.
5. Sackett D, Rosenberg W, Gray J, Haynes R,
Richardson W. Evidence based medicine: what it
is and what it isn’t. Br. Med. J. 1996; 312: 71–2.
6. Canadian Coordinating Office for Health Tech-
nology Assessment. Guidelines for economic
evaluation of pharmaceuticals: Canada. 2nd ed.
Ottawa: Canadian Coordinating Office for Health
Technology Assessment (CCOHTA); 1997.
7. Commonwealth Department of Health, Housing
and Community Service. Guidelines for the
pharmaceuticals industry on the submission to the
Pharmaceutical Benefits Advisory Committee.
Canberra: Australian Government Publishing
Service, 1992.
8. National Institute for Clinical Effectiveness:
Appraisal of new and existing technologies:
Interim guidelines for manufacturers and sponsors.
December 1999, London: NICE http://www.nice.
org.uk/appraisals/appraisals.htm.
9. Antman E, Lau J, Kupelnick B, Mosteller F,
Chalmers T. A comparison of results of meta-
analyses of randomised control trials and recom-
mendations of experts. JAMA 1992; 268: 240–8.
10. Chalmers I, Altman D. Systematic Reviews.
London: BMJ Publishing, 1995.
5
Introduction

Part I
CLINICAL TRIALS
INTRODUCTION by RICHARD J. LILFORD
and ANDREW STEVENS
The concept of control groups is mentioned as
early as the book of Daniel and was formalised
by Sir Francis Bacon in the 17th century. James
Lind conducted a famous trial of different meth-
ods of prophylaxis against scurvy, and discov-
ered the benefits of fruit for sailors.1
This was
not a randomised experiment, and although the
idea of allocation of therapy by ‘lot’ is an old
one, practical application is really a 20th century
phenomenon. The statistics were worked out by
Ronald Fisher in an agricultural context, and a
landmark randomised trial in humans was Sir
Austin Bradford Hills’ study of streptomycin
treatment for tuberculosis, conducted during the
Second World War.2
The Randomised Controlled Trial (RCT) is
the archetypal primary research method of
Health Technology Assessment (HTA). About a
third of a million RCTs are registered on the
Cochrane database of controlled clinical trials,
and this list falls far short of covering the entire
literature base on RCT methodology.3
RCTs
have given some very clear pointers to practice:
clot busting and antiplatelet drugs are effective
in heart attack, multi-agent chemotherapy delays
the onset of AIDS among carriers of HIV virus,
and antibiotics are effective in surgery where a
contaminated viscus is breached. Some treat-
ments have such spectacular effects that their
effectiveness can be established without the
need for RCTs – volume replacement for
massive haemorrhage, anti-D to prevent rhesus
sensitisation in newborns, and use of the supine
position to prevent ‘cot death’. However, most
advances in medicine are less spectacular –
moderate gains which collectively contribute to
major overall improvements in welfare. These
modest but worthwhile gains cannot be detected
easily – unlike the few massive breakthroughs.
Moderate biases may obscure or exaggerate
such gains or even create the impression of
benefit in the face of harm. RCTs, blinded when
possible, provide protection against these mod-
erate but relatively important biases.4
Random-
isation provides protection against selection bias
– inaccuracy that occurs if patients in treatment
and control groups have different capacities to
benefit. Randomisation, provided that the alloca-
tion is fully concealed, ensures that confounding
variables are distributed by chance alone. Blind-
ing prevents the possibility of performance bias
(augmented or impaired outcome resulting from
administration of different treatments, apart
from the treatment of interest, to study or con-
trol patients) and outcome bias (distorted out-
come, involvement or interpretation in the light
of hope and expectation).
It has been shown empirically that random-
ised controlled trials in general produce more
modest estimates of treatment effects than his-
torically controlled studies. Good RCTs – studies
which obey the established scientific criteria –
produce different results from poorly conducted
studies. Well-conducted RCTs are, it is widely
agreed, the most reliable method for comparing
most health technologies.5
There may be ques-
tions about their external validity (generalis-
ability) and about their practicability in various
contexts, but they remain the bedrock of much
HTA.

Randomisation is a simple concept, but RCTs
have spawned a wide range of complex issues
that challenge those who commission and con-
duct applied research. Large simple trials have
led to some very important discoveries such as
the value of magnesium in the treatment for
eclampsia. However, the range of possible inter-
actions (and hence of potential interactions
between interventions) is rising exponentially,
and it will not be possible to answer all research
questions using the large simple philosophy.
Factorial designed studies (where patients are
re-randomised to different therapies), families of
trials (where patients are offered different trials
at different stages of their disease according to
eligibility), and trials nested within prospective
frameworks for collection of routine data, will
all be necessary. Some therapies are given to
groups of people, not individuals, and this calls
for cluster trials while at the other end of the
scale, patients may act as their own controls in
n 5 1 trials. Some interventions are complex
‘packages’ made up of many components, where
measurement of ‘fidelity’ to the intended pack-
age is useful to those who may wish to adopt the
results. Trials of rapidly evolving technologies
are an increasing concern given the pace of
technological development – the question is
when to start such studies, and one approach is
to offer randomisation at an early stage, accept-
ing that the technology might continue to
improve for some time after the start of the
study. Another (often related) problem arises
from interventions whose effects are critically
dependent on practitioner skill or organisational
factors – one approach is to ensure recruitment
from a very broad sample, and another is to
randomise both patient and practitioner. All
these refinements have cost implications, and a
compromise may be necessary between the ideal
and the achievable, in this as in other aspects of
life. In short, the study of trial design is still
evolving.
This section deals firstly in the ethics of
RCTs. A general review and discussion of the
ethics of RCTs, by Lilford and colleagues con-
siders the main philosophical positions and
shows that from two points of view – utilitarian
and deontological – the essential requirements
are that the treatments being compared should
be an equal bet in prospect, the situation of
patient equipoise where the expected utilities of
each treatment on offer are equivalent.6
Clearly
the competent patient is in the best position to
make this decision, given available ‘prior’
probabilities; hence there is no conflict between
maximising welfare and respecting autonomy, in
this area of ethics at least. In the subsequent
chapter, Ashcroft shows how these principles
may be interpreted in different moral traditions
and considers whether the doctrine of informed
consent – so crucial in the West – may be
viewed somewhat differently in cultures where
an egalitarian relationship between professional
and patient is less widely accepted.
Ross and colleagues then discuss the quality
of trials. The factors impeding recruitment are
classified as those affecting clinicians’ willing-
ness to collaborate in a study, their willingness
to offer patients entry and patients’ willingness
to participate, given the offer. Patients seem less
willing to participate in trials where the compar-
ator treatments are very different in their nature
and immediate effects – surgery versus radio-
therapy for prostate cancer is a recent example.
The importance of separating the ‘business’ of
running trials from their conception and instiga-
tion is stressed. This ensures that those who may
have an emotional or pecuniary interest in the
results of a trial are kept at arm’s length from
the processes and analyses which may bias the
results – for example, interference in the ran-
domisation sequence is prevented.
However, a strong recommendation from the
authors is that trials should be kept simple so
that logistic difficulties do not act as a strong
disincentive to participation. Reporting of results,
so that the link between outcome and allocated
treatment is clear, is also discussed in this chap-
ter and the agreed methodology for reporting
studies (CONSORT) is emphasised.7
O’Connell and colleagues then consider the
problem of relating results from randomised
trials to patients in another time and place. Most
authors address these issues by describing the
inclusion and exclusion criteria, subgroup analy-
sis and (less often) comparison of trial partici-
pants with those not in the trial. However, there
are strong arguments for more individualised
assessment of benefits and harms. Clinical trials
can be thought of as a method to produce good
empirical data with which to populate risk/
benefit models, in which probabilities of good
and bad outcomes, and the values attached to
these outcomes are made explicit. There is also
a great need for more empirical work on the
transferability of results from one group of
patients to another, especially in situations
where there is no obvious reason why they
might differ. For example, there may be no prior
reason to expect results to vary by sex, race or
age. It is difficult to know what to do when
statistically significant subgroup effects arise.
An essential principle, however, is that even if
relative risks are fixed across different sub-
groups, absolute risks may vary. This is where
modelling comes into its own, since it enables
results to be extrapolated to those with a differ-
ent base-line prognosis.8

Lastly, those who think they understand the
‘placebo effect’ should read the section by
Crowe and Colleagues. Placebo effects result
largely from expectation and are psychologically
mediated. Placebos, however, are not merely
inert substances, but include many other aspects
of treatment such as practitioners themselves,
operations, injections, and so on. An important
conclusion to derive from their study is that the
placebo effect, as described above, is an import-
ant and ineluctable component of many ther-
apies. It follows, therefore, that placebo effects
should be controlled out only when they are an
artefact of the research design, and not when
they would continue to operate in everyday
practice. Our current concern is that although
the placebo effect is very real in non-trial prac-
tice, when patients have no reason to doubt that
they are receiving the putative active ‘ingredi-
ent’, this is less certain in the context of the trial,
where there is only a say, 50%, chance that
the active ingredient is included. The role of
the placebo both inside and outwith trials con-
tinues to be intriguing and more research is still
needed.
REFERENCES
1. Bull JP. The historical development of clinical
therapeutic trials. J. Chron. Dis. 1959; 10(3):
218–48.
2. Hill AB. Statistical Methods in Clinical and
Preventive Medicine, London: Oxford University
Press 1962.
3. Cochrane Controlled Trials Register, The Cochrane
Library, 1999, Oxford: Update Software Ltd.
4. Chalmers I, Sinclair JC. Promoting perinatal
health: is it time for a change of emphasis in
research? Early Human Development 1985; 10:
171–91.
5. Peto R, Collins R, Gray R. Large-scale randomized
evidence: large, simple trials and overviews of
trials. Ann. N. Y. Acad. Sci. 1993; 703: 314–40.
6. Lilford RJ, Jackson JC. Equipoise and the ethics of
randomisation. J. R. Soc. Med. 1995; 88: 552–9.
7. Altman D. Better reporting of randomised con-
trolled trials: the CONSORT statement. Br. Med. J.
1996; 313: 570–1.
8. Lilford RJ, Pauker SG, Braunholtz D, Chard J.
Decision analysis and the implementation of
research findings. Br. Med. J. 1998; 317: 405–9.
9
Part I: Clinical Trials

1
Ethical Issues in the Design and Conduct
of Randomised Controlled Trials
R I C H A R D J . L I L F O R D , S A R A H J . L . E D W A R D S ,
D A V I D A . B R A U N H O L T Z , J E N N I F E R J A C K S O N ,
J I M T H O R N T O N and J E N N Y H E W I S O N
SUMMARY
We have reviewed the literature on the ethics of
clinical trials, and found that there is little agree-
ment about what constitutes moral justification
for trials. However, a widely held and defensible
view, distilled from both utilitarian and Kantian
ethics, is as follows: randomised trials are
ethical, from the doctor’s and patient’s point of
view, when competent patients have been fully
consulted so that their values determine what is
in their best interests and they have consented to
trial entry. Decision theory tells us that, in the
more usual situation where trial treatments are
freely available, patients must be in a state of
equipoise (alternative and available treatments
have equal expected utilities) before they con-
sent to randomisation. Patients may expect to
increase their expected utility by being altruistic,
making altruism a discrete factor in its own
right.
The interests of patients who cannot give their
consent (because of incompetence, say) should
be protected, and so altruism should not be
assumed. A surrogate, perhaps appointed by the
patient themself, must provide the values, using
average values for example or, in the case of
advance directives, previously elicited values.
In the less usual situation where a treatment is
only available in a clinical trial, then randomisa-
tion is ethical, provided that patients are in
equipoise or better, i.e. provided the restricted
treatment is expected to be at least as good as
what is available routinely.
Equipoise is a more exacting requirement for
the ethical conduct of clinical trials than is
‘uncertainty’. It is important that participants are
not deceived by doctors equivocating on the
term, as this limits their capacity to give an
informed choice.
The offer of entry into a clinical trial is not
itself psychologically upsetting for the majority
of patients. Patients cope best with the invitation
to join the clinical trial when they have good
overall understanding of the consent material. A
number of deliberate experiments have com-
pared different methods for inviting people to
participate in clinical trials, and these have
shown that giving more information tends to
result in greater understanding, although fewer
patients then accept randomisation.
There may be some clinical benefit from
participating in a clinical trial, even when the
trial itself produces a null result. In particular,
when there is a pre-existing effective treatment
whose effectiveness is enhanced in the trial by
the use of protocol or by a Hawthorne effect.
Since the Helsinki declaration seeks specifically
to reassure patients that their care will not be
affected in a detrimental sense if they do not take
part in a clinical study, this putative trial effect
cannot be used as any form of inducement.
The requirement of equipoise (or better) has a
number of implications for RCT practice and
design.

1 Under-powered trials are not necessarily
unethical in their ‘use’ of patients.
2 Interim analyses are desirable, although the
current practice of keeping preliminary data
secret is suspect, especially when trial treat-
ments are widely available.
3 Adaptive designs do not solve the problem
of failing to avoid reduced expected utilities
for each participant when interim evidence
favours a treatment which could be selected
outside the trial.
4 Placebo-controlled trials generally are not
usually appropriate when proven effective
treatments are routine, and those using
‘active’ placebos with adverse effects to
mimic the intervention for the preservation
of blinding should be used with caution, if at
all.
5 Cluster RCTs can be categorised into
cluster–cluster and individual–cluster trials,
depending on the type of intervention under
study. Guardians should consent to random-
isation in both types of trial. The best inter-
ests of the cluster is of paramount concern
in the former type, while individuals can
consent to any intervention in the latter type
of trial.
6 Many behavioural interventions may con-
taminate controls, either directly through
fully informed consent or indirectly through
close contact with and communication
between experimental and control individ-
uals. Zelen randomisation, where consent is
typically sought only from the intervention
group after randomisation, may be a good
compromise in such cases.
Randomised controlled trials have always pro-
voked debate and discussion about ethical con-
cerns because of a variety of issues regarding
possible conflicts of interest between society and
the individual patient. This chapter sets out these
various ethical concerns. We first carried out a
systematic search of the literature, relating to
the ethics of conducting clinical trials, which
included empirical studies to inform and pos-
sibly substantiate such discussion. Full details of
our search strategy can be found in a recent
Health Technology Assessment monograph.1
We then abstracted the main issues and argu-
ments, together with any informative empirical
data. This material was synthesised using an
intellectual framework, a product of which is the
following narrative focusing on the issue of
epistemological uncertainty as justification for
entering patients in trials. The idea of uncer-
tainty or, less ambiguously, equipoise was then
applied to a number of different methodological
and practical situations, and conclusions are
offered based on this analysis.
EPISTEMOLOGICAL UNCERTAINTY AS
JUSTIFICATION FOR RECRUITING PATIENTS
Introduction
During this century, the medical profession has
acquired a growing capacity to distinguish
between what they know and what they do not,
thereby changing the professional emphasis
from demanding public trust without sound
epistemological basis to an ‘evidence based’
approach to medical practice. This is typically
contrasted with practice based on simple
intuition. Although the two approaches may lead
to the same answer in the end, scientific knowl-
edge provides explicit justification for belief.
The Randomised Controlled Trial (RCT) is
deemed the most reliable scientific method in
medicine today by virtue of its unique capacity
to eliminate selection bias. However, there is an
apparent moral tension between serving the
interests of society at large, and those of individ-
ual participants.
Societal interests
The main reason for using the RCT design is a
scientific one, for properly conducted RCTs pro-
duce the most valid data from which society can
benefit.2
Medical research advances medical
practice and hence goes some way towards
alleviating suffering and avoiding harm. The
RCT is not always scientifically necessary or
desirable, but it is most useful when expected
treatment effects are small, yet worthwhile.
It is often thought that any RCT rests exclus-
ively – and some would say precariously – on
classical utilitarian justification for its moral
acceptability.3
Society needs the good quality
evidence which RCTs offer. However, utilitar-
ianism is not the only moral theory from which
the use of RCTs in medical research can draw
support. Indeed, Kantians see individuals as
having tacit obligations to society at large, too.
Not all RCTs are morally justified, however
valuable to society they may seem, and this
chapter is concerned with the need to advance
medical knowledge, whilst protecting those indi-
viduals who are recruited at the same time.
Indeed, failure to protect individual’s interests
and their rights is unlikely to increase social
value in the long term, and patients may lose
trust in the profession and then reject trials
altogether, the burden of involuntary and per-
petual altruism being too great to bear.
Individual interests
Perhaps the most common criticism to be lev-
elled at RCTs is that they are insensitive to the

best interests of current patients who participate
in them and this insensitivity is at odds with the
model doctor–patient relationship.4–6
We will
see that, under certain conditions, the societal
interest can be promoted without threat to the
individual’s own best interests.
In this chapter, we will analyse the idea that
the tension between societal and individual
interests can be resolved by making recourse to
the epistemological ‘uncertainty’ principle.7
Put
simply, if the clinician does not know which
treatment is best, then the patient is not dis-
advantaged in prospect by being randomised and
society benefits from the knowledge gained.
However, it is necessary to clarify what ‘knowl-
edge’ and ‘best interests’ mean before they can
be properly lodged with ethical theory and then
operationalised by applying them to practical
difficulties with trials and their design.
Knowledge
Dealing first with ‘knowledge’, consider first
two treatments (A and B) with equivalent (or
no) side-effects. Knowing can have two mean-
ings: firstly, that there is no preference between
two treatments, since they have identical side-
effects and A is as likely to be better than B or
vice versa. Secondly, that there is some prefer-
ence in the sense that one of the treatments is
more likely to be superior, but this is uncertain;
treatment A has not been ‘proven’ to be more
effective than B. Indeed, Gifford observes that
most arguments which use uncertainty as justifi-
cation for RCTs equivocate on the term ‘know’,
sometimes giving the first interpretation above,
and sometimes the second.8
Uncertainty as the
opposite of certainty, allows a clinical trial to
proceed in the face of considerable preference.
However, if there is ‘absolute’ uncertainty, the
decision-maker is ‘agnostic’ or ‘in equipoise’ –
that is, the benefits in prospect for both treat-
ments are equal, and the outlook for the patient
is the same, whether or not a trial is conducted.9
For those familiar with Bayesian thinking, this
situation would arise where the point of balance
of the ‘prior’ for the difference A 2 B was zero.
Best Interests
We need to clarify the meaning of ‘best’ in the
context of the clinician’s obligation to give
patients the ‘best’ treatment. Two situations
apply. Firstly, the above situation where treat-
ments A and B (two treatments are discussed
here for convenience) have equal side-effects in
prospect, and the major outcomes are binary
(e.g. live or die). In this situation, A is preferred
to B, provided it is more effective (a situation
often referred to as ‘probabilistic dominance’)
because all values are equal to unity. Under
these circumstances, equipoise exists if the
expected effectiveness of A and B are equal, or,
more formally (and in Bayesian notation) if the
‘priors’ balance on the same point.
However, we must acknowledge that treat-
ments typically have unequal side-effects a
priori.9
A, for example, might be mutilating
surgery, while B is a smaller operation or an
inexpensive drug with few side-effects. In order
for the treatments to be equally desirable under
these conditions (i.e. for equipoise to exist), A
must be better than B, in prospect, on the major
outcome so that superior effectiveness can be
traded off against worse side-effects. In decision
analytic language, the expected utilities of A and
B should be the same; the expected advantages
of A on the main outcome measure are precisely
offset by the lower side-effects of B. Equipoise
under these circumstances occurs when A is
expected to be superior (on the main outcome
measure) to B. If the expected difference in
outcome equals, but does not exceed the trade-
off that individual patients require, then the
patient is in individual equipoise. Of course,
there may be many outcomes where the relative
effects of the treatment are uncertain. However,
this does not change the fundamental point, that
a decision analysis can be constructed around
certain effects (such as the mutilation of sur-
gery), uncertain effects (for which ‘prior’ prob-
abilities exist, albeit within wider credible inter-
vals), and the values that might be ascribed to
these various outcomes. If we leave aside, just
for a moment, any potential negative or positive
effects that the trial itself might have, we may
argue that the patient is not disadvantaged
relative to routine care, provided the expected
utilities of the comparative treatments are
equivalent. The subject of decision analysis – or
expected utility theory – is fully described
elsewhere.10
Equipoise: Collective or Individual?
The term equipoise was made popular by Freed-
man in a landmark article in the New England
Journal of Medicine,7
and we argue that its
theoretical meaning can be made explicit within
a Bayesian and decision analytic framework.
Freedman distinguished between two kinds of
equipoise – collective and individual. By col-
lective equipoise, he meant that a society of
experts (say clinicians contemplating a particu-
lar trial) do not know which of two or more
treatments is best, although clinicians within the
community may have different ‘hunches’. By
individual equipoise, he meant that a particular
13
Ethical Issues in Design and Conduct of RCTs

clinician was equipoised, or that he had no
hunch whatsoever. Using our definition would
mean that this clinician regards the treatments as
having equal expected utilities. Freedman goes
on to argue that ethically collective equipoise
is a sufficient condition for recruiting patients
to trials, with the proviso that they consent,
irrespective of whether the individual clinician
in question does or does not have a prefer-
ence. He starts by pointing out that individual
equipoise is a fragile state which might be very
hard to find, given the exacting requirement
for the interaction between ‘prior’ probabilities
and values to produce precisely equal expected
utilities across the comparator treatments.
Because individual equipoise is such an elusive
state in practice, Freedman argues, it can be
discounted as too stringent a moral basis for
trial entry. When individuals are equipoised,
this is simply a bonus, not a requirement. As
scientists, we would like to go along with Freed-
man, but we are conscious of serious flaws in his
argument.
Problems with Freedman’s Argument
The first point that Freedman ignores is that
collective equipoise is seldom evenly balanced.
It, too, is a fragile state, only now among a
group of people. Typically, the numbers of
clinicians who might have preferences (hunches)
for one treatment or another will be unequal in a
ratio of, say, 30 to 70. Freedman does not say
how widespread a hunch can be before col-
lective equipoise collapses into consensus over
which treatment is best. Just how many
clinicians need to prefer the trial treatment
before a trial of it becomes unethical? This was
investigated by Johnson and colleagues,11
who
conducted an ethnometric study to identify the
threshold of collective preference beyond which
a trial would be unacceptable by members of the
public, and showed that, once the ratio of num-
ber of clinicians for the trial treatment to the
number of clinicians against it went beyond
70:30, the great majority of respondents would
regard a trial of a potentially life-saving inter-
vention as unethical.
However, there are yet further problems with
Freedman’s argument, i.e. that where collective
equipoise and individual equipoise do not co-
incide, collective equipoise is a sufficient moral
basis upon which to recruit patients. He argues
that the obligation of a doctor to a patient is
different from those which, say, a parent has
towards a child – Freedman thus argues that
clinicians need not adhere to the ‘gold standard’,
whereby one should not randomise a patient
unless one was prepared to do the same for a
beloved family member. Freedman defends his
position on the grounds that doctors are judged
by, and should behave according to, collective
norms.
However, this does not say how the doctor is
supposed to operationalise the concept of col-
lective versus individual equipoise – is the doc-
tor to tell the patient only that there is collective
uncertainty about the best treatment while keep-
ing any personal preference under wraps?
Besides, couching consent only in terms of
collective uncertainty and not individual equi-
poise leaves open the very real possibility that
the patient will interpret this to mean that the
clinician is personally equipoised, whereas the
clinician might mean only that the usual statis-
tical conventions of ‘certainty’ have not been
fulfilled. Equivocation can lead to deception.12
Failure to offer to disclose any such preference
at a minimum is an economy with the truth, and
tantamount to deception. Deception, intentional
or otherwise, is a limit to autonomous decision-
making13,14
and so Freedman’s allegiance to
collective equipoise does not sit well with his
position on autonomy: like most modern authors
he strongly asserts the need to respect individual
choices. Such autonomy is exerted through dif-
ferent people manifesting their own personal –
and often very different – value systems. In
Western cultures, it is a reasonable assumption
that individuals are likely to have different
values, perhaps as a result of their freedom to
choose how to live. People may make very
different trade-offs. Given this, it follows that
there can be no such thing as collective equi-
poise in any decision involving a trade-off –
how can it simultaneously be argued that
collective equipoise provides sufficient moral
cover for clinical trials and that autonomy is a
fundamental principle, when the latter implies
that two different people, confronted with
exactly the same scenario, may make very dif-
ferent choices! On the basis of the best ‘prior’
probability estimates, one person may choose
treatment A, another treatment B, while a third
might be equipoised between A and B and hence
willing (or eager) to be randomised. Therefore,
we think that anybody who holds autonomy as a
central principle must also regard equipoise as
the property of the patient. It arises when prior
probability estimates and an individual patient’s
values interact in such a way as to produce equal
expected utilities.
There is near-universal agreement that the
‘competent’ patient should be the source of
these values.1,15,16
Probabilities are normally the
province of the care provider who has a moral
obligation to bring as much knowledge and
experience to bear on the problem as possible
under the constraints of practice. In some cases,

Other documents randomly have
different content

law in creating the Project Gutenberg™ collection. Despite these
efforts, Project Gutenberg™ electronic works, and the medium
on which they may be stored, may contain “Defects,” such as,
but not limited to, incomplete, inaccurate or corrupt data,
transcription errors, a copyright or other intellectual property
infringement, a defective or damaged disk or other medium, a
computer virus, or computer codes that damage or cannot be
read by your equipment.
1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES - Except
for the “Right of Replacement or Refund” described in
paragraph 1.F.3, the Project Gutenberg Literary Archive
Foundation, the owner of the Project Gutenberg™ trademark,
and any other party distributing a Project Gutenberg™ electronic
work under this agreement, disclaim all liability to you for
damages, costs and expenses, including legal fees. YOU AGREE
THAT YOU HAVE NO REMEDIES FOR NEGLIGENCE, STRICT
LIABILITY, BREACH OF WARRANTY OR BREACH OF CONTRACT
EXCEPT THOSE PROVIDED IN PARAGRAPH 1.F.3. YOU AGREE
THAT THE FOUNDATION, THE TRADEMARK OWNER, AND ANY
DISTRIBUTOR UNDER THIS AGREEMENT WILL NOT BE LIABLE
TO YOU FOR ACTUAL, DIRECT, INDIRECT, CONSEQUENTIAL,
PUNITIVE OR INCIDENTAL DAMAGES EVEN IF YOU GIVE
NOTICE OF THE POSSIBILITY OF SUCH DAMAGE.
1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If you
discover a defect in this electronic work within 90 days of
receiving it, you can receive a refund of the money (if any) you
paid for it by sending a written explanation to the person you
received the work from. If you received the work on a physical
medium, you must return the medium with your written
explanation. The person or entity that provided you with the
defective work may elect to provide a replacement copy in lieu
of a refund. If you received the work electronically, the person
or entity providing it to you may choose to give you a second
opportunity to receive the work electronically in lieu of a refund.

If the second copy is also defective, you may demand a refund
in writing without further opportunities to fix the problem.
1.F.4. Except for the limited right of replacement or refund set
forth in paragraph 1.F.3, this work is provided to you ‘AS-IS’,
WITH NO OTHER WARRANTIES OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.
1.F.5. Some states do not allow disclaimers of certain implied
warranties or the exclusion or limitation of certain types of
damages. If any disclaimer or limitation set forth in this
agreement violates the law of the state applicable to this
agreement, the agreement shall be interpreted to make the
maximum disclaimer or limitation permitted by the applicable
state law. The invalidity or unenforceability of any provision of
this agreement shall not void the remaining provisions.
1.F.6. INDEMNITY - You agree to indemnify and hold the
Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and
distribution of Project Gutenberg™ electronic works, harmless
from all liability, costs and expenses, including legal fees, that
arise directly or indirectly from any of the following which you
do or cause to occur: (a) distribution of this or any Project
Gutenberg™ work, (b) alteration, modification, or additions or
deletions to any Project Gutenberg™ work, and (c) any Defect
you cause.
Section 2. Information about the Mission
of Project Gutenberg™

Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new
computers. It exists because of the efforts of hundreds of
volunteers and donations from people in all walks of life.
Volunteers and financial support to provide volunteers with the
assistance they need are critical to reaching Project
Gutenberg™’s goals and ensuring that the Project Gutenberg™
collection will remain freely available for generations to come. In
2001, the Project Gutenberg Literary Archive Foundation was
created to provide a secure and permanent future for Project
Gutenberg™ and future generations. To learn more about the
Project Gutenberg Literary Archive Foundation and how your
efforts and donations can help, see Sections 3 and 4 and the
Foundation information page at www.gutenberg.org.
Section 3. Information about the Project
Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-
profit 501(c)(3) educational corporation organized under the
laws of the state of Mississippi and granted tax exempt status
by the Internal Revenue Service. The Foundation’s EIN or
federal tax identification number is 64-6221541. Contributions
to the Project Gutenberg Literary Archive Foundation are tax
deductible to the full extent permitted by U.S. federal laws and
your state’s laws.
The Foundation’s business office is located at 809 North 1500
West, Salt Lake City, UT 84116, (801) 596-1887. Email contact
links and up to date contact information can be found at the
Foundation’s website and official page at
www.gutenberg.org/contact

Section 4. Information about Donations to
the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission
of increasing the number of public domain and licensed works
that can be freely distributed in machine-readable form
accessible by the widest array of equipment including outdated
equipment. Many small donations ($1 to $5,000) are particularly
important to maintaining tax exempt status with the IRS.
The Foundation is committed to complying with the laws
regulating charities and charitable donations in all 50 states of
the United States. Compliance requirements are not uniform
and it takes a considerable effort, much paperwork and many
fees to meet and keep up with these requirements. We do not
solicit donations in locations where we have not received written
confirmation of compliance. To SEND DONATIONS or determine
the status of compliance for any particular state visit
www.gutenberg.org/donate.
While we cannot and do not solicit contributions from states
where we have not met the solicitation requirements, we know
of no prohibition against accepting unsolicited donations from
donors in such states who approach us with offers to donate.
International donations are gratefully accepted, but we cannot
make any statements concerning tax treatment of donations
received from outside the United States. U.S. laws alone swamp
our small staff.
Please check the Project Gutenberg web pages for current
donation methods and addresses. Donations are accepted in a
number of other ways including checks, online payments and

credit card donations. To donate, please visit:
www.gutenberg.org/donate.
Section 5. General Information About
Project Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could
be freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose
network of volunteer support.
Project Gutenberg™ eBooks are often created from several
printed editions, all of which are confirmed as not protected by
copyright in the U.S. unless a copyright notice is included. Thus,
we do not necessarily keep eBooks in compliance with any
particular paper edition.
Most people start at our website which has the main PG search
facility: www.gutenberg.org.
This website includes information about Project Gutenberg™,
including how to make donations to the Project Gutenberg
Literary Archive Foundation, how to help produce our new
eBooks, and how to subscribe to our email newsletter to hear
about new eBooks.

Welcome to Our Bookstore - The Ultimate Destination for Book Lovers
Are you passionate about books and eager to explore new worlds of
knowledge? At our website, we offer a vast collection of books that
cater to every interest and age group. From classic literature to
specialized publications, self-help books, and children’s stories, we
have it all! Each book is a gateway to new adventures, helping you
expand your knowledge and nourish your soul
Experience Convenient and Enjoyable Book Shopping Our website is more
than just an online bookstore—it’s a bridge connecting readers to the
timeless values of culture and wisdom. With a sleek and user-friendly
interface and a smart search system, you can find your favorite books
quickly and easily. Enjoy special promotions, fast home delivery, and
a seamless shopping experience that saves you time and enhances your
love for reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!
ebookgate.com

The advanced handbook of methods in evidence based healthcare Brazier

More Related Content

Similar to The advanced handbook of methods in evidence based healthcare Brazier (20)

Recently uploaded (20)

The advanced handbook of methods in evidence based healthcare Brazier