Using primary care databases to evaluate drug benefits and harms: are the results replicable and valid?
1 like1,168 views
The document discusses the validity and replicability of studies using primary care databases (PCDs) to evaluate drug benefits and harms. It presents examples of replication studies on statins and beta-blockers, highlighting both striking similarities and significant differences in outcomes. The author emphasizes the challenges of conducting PCD replications, including data quality issues and the need for transparent reporting of methodologies.
Using primary care databases to evaluate drug benefits and harms: are the results replicable and valid?
- 1. Using primary care databases to evaluate drug benefits and harms: are the results valid and replicable? David A. Springate, University of Manchester Centres for Primary Care/Biostatistics
- 2. Outline 1. Primary Care Database (PCD) study validity 2. PCD Replications – Statins and Ischaemic heart disease – β-blockers and Cancer 3. Lessons to be learned
- 3. PCD studies are all the rage. . . Number of UK PCD publications is rapidly increasing 1990 1995 2000 2005 2010 050100150 PCD articles in pubmed year Numberofarticles There is global interest in UK PCD research Institutions affiliated with UK PCD publications xx x x x xxxxxx x x x x xx x x x xx x xx x xxx xx xxx x xx xxxxxx xx xx x xx x x x xx x x x xx x x xxxxxx x x x x x x xx x xxx x xxxxx xxx xxx x x x x xx xxx xx x xx x x xxx xx x x x x x x x x x x xx xx x xxx x x x x x x x x xx xx x xx xxxx x x x x xx x xx x xxx x xx xx xx x x xx x xxxxxx x x x xx x xxx x x xxx x x x x xxxxx x xx x xx xxxxxx xx xx x x x x xx x xxx x x xx xx x xxxxx x xxxxx x xx xxx x x x xx xx xxx x x x x xx x xx xx x x x x xx xx xxxxxx x x x xx x x x x x x x xx x x x x x xx x x x xxx x xxxxx x x x xxxx x x xxxxx xx xx x x xxxxxxxx xxx x xxxxx x x xx xxx x x xx xxxx x x xxx xx x xx xxxxx x xx x x xx x x x xx x x x xxx x xx x xxxx xx xxx xx x xx x xx x xxxx xxx x x xxx x x xxxx x x x x x x x x x x x x x x xxxxxx xxxx xx x xxx x x x x x x x x xx x x x x x xxx x x x xx x xxx x x x x x x x xx x x x x x x xx xx xxxx x x x x xxx x x xx xxx x x xxx x x x xx xxxxxxxxx xx xx x xxxx xx x xxxx x x xx x x x x xx x xxx x xx xxxxxx xx x xx x x x xxx x x x xxxxx xx xx x x x x x x x x x x x xxxxx x xx xxx x xxx x x x x x x x xx x x x x x xxx xx x xxxxx x x xx xx x x x xxxxxxx xx x x xxxx xx xx x x x x xxxx xx xx xxx xxx x xx xx x xxx x x x x x xxx x x x x xxxx x x x x xxxx xxx xxxxxxxx x xx xx xx x xxxx x x x xxxx x x x xx xxxx xx x xx xxx xxx x x x xxx xxxx xxxx x xx x x x x xx x x x x xx xxx x x x x x x x x xxxxxxxx x xxx xx x xxx x xx xxxx xx x xxxxxxxx xxxxx x xx xx x xxxxxxx x x xx xxx x x xx x xx xx x x xx x x x xxx x x x xxx x x xx xx xx xxx x x x xx x x xxx x x x x xx x x xxx x xx xxxxxxxx x x x x x x xx x xxxxxx x x xxxx xxx x xxx x x x x xx x x xx x x x x x x x x x x x x x x xx x xx xx x x x xx x x x xxx x xxx x xx xx x x x x x xx xx xx x x xx x xxxxxxx x xxxxxxxxxxx xxxxxxxxxxx xxx x x x xxxx x xxxx xxxxxxxxxxxxxxxxxx xxxxx
- 4. Uses of Primary care databases. . . • Prevalence / incidence studies • Associations between conditions • Harms and risks of treatments • Comparative effectiveness • RCT comparisions / replications (and replacements?)
- 5. BUT, There are still concerns about the validity of PCD-based studies. . . Threat Refs Data quality Herrett 2009, Khan 2010, Jordan 2004 Data completeness Marston 2010, Delaney 2007, Collins 2010 Confounding Tannen 2008, Lewis 2007 Clinical coding www.ClinicalCodes.org
- 6. PCD Replications “Non-reproducible single occurrences are of no significance to science.” —– Karl Popper (1959) An approach to validity that asks whether flaws and differences in the data make any difference to the ultimate conclusions rather than looking at validity and completeness of the underlying individual data http://xkcd.com/242
- 7. Replicating studies in another, independent PCD • Agreement implies that conclusions are not dependent on data source • Some factors could influence both (such as confounding by indication) • First completely independent PCD replication (but see Vinogradova (BMJ 2013): relationship between bisphosphate exposure and cancer in QResearch and CPRD)
- 8. Replications were performed in CPRD • Largest UK primary care database (CPRD-GOLD) • ˜ 14 million patients • ˜ 650 practices across the UK • Uses the Vision GP computer system
- 9. Criteria for replication 1. Effectiveness studies 2. Different GP computer system from CPRD (not Vision) 3. No practice overlap with CPRD 4. Representative coverage 5. Primary Care Database (Not integrated primary/secondary/pharmacy)
- 10. PCD replications Hippisley-Cox and Coupland (2006) Effect of statins on the mortality of patients with ischaemic heart disease: population based cohort study with nested case-control analysis. Heart 92:752-758 (QResearch) Shah, Carey et al. (2011) Does β-adrenoceptor blocker therapy improve cancer survival? Findings from a population-based retrospective cohort study Br J Clin Pharmacol 72:157-161 (DIN-LINK) We then compare • summary statistics • mortality rates • model coefficients and standard errors
- 11. Effect of statins on the mortality of patients with ischaemic heart disease (QResearch) Objective To measure the effect of statins on mortality for community based patients with IHD Design Cohort survival analysis and nested case-control Setting 1.18 million patients in 89 practices Subjects Patients with first diagnosis of IHD between January 1996 and December 2003 Outcomes - Cohort: Adjusted hazard ratios (+/- 95%CI) for all-cause mortality - Case-control: Odds ratio (+/-95%CI) for current use of statins, previous use and duration of use
- 12. Effect of statins on the mortality of patients with ischaemic heart disease Summary statistics Measure Analysis CPRD Qresearch Number of practices Cohort 661 89 Number of patients Cohort 91589 13029 Cases 15591 2266 Controls 62356 9064 Median age Cases 80 80 Controls 79 80 Percent female Cases 45.5% 44.3% Controls 45.5% 44.3% Median followup (months) Cases 22.1 20.3 Controls 22.5 21 Percent on statins Cases 17% 19.6% Controls 23.6 25.4%
- 13. Mortality rates for patients with IHD Age Comorbidity Sex 0 100 200 300 400 Total0−4445−5455−6465−7475−8485−94 95_plus N o diabetes D iabetes N o hypertension H ypertensionN o C C F C C F Fem ale M ale Patient group Mortalityrate(per1000personyears) PCD CPRD QResearch
- 14. Survival analyses for patients on Statins QResearch Adjusted HR 0.47 (0.41 to 0.53) CPRD 0 2 4 6 8 Time since diagnosis of IHD (Years) Survival 0.000.250.500.751.00 Adjusted HR 0.43 (0.40 to 0.46)
- 15. Case-control analysis: Odds ratios for effects of Statins on mortality in IHD patients Odds are relative to patients not on statins. Dotted line represents 1:1 odds Previously on statins Currently on statins 0.0 0.5 1.0 1.5 2.0 AllStatins Atorvastatin C erivastatinFluvastatinPravastatinSim vastatin AllStatins Atorvastatin C erivastatinFluvastatinPravastatinSim vastatin Statin type Oddsratio PCD CPRD Qresearch
- 16. Misleading pooled odds ratios Combining effects of e.g. two drugs of the same BNF chapter Group Y N Odds ratio Drug 1 Cases 600 200 3/3 = 1 Drug 1 Controls 75 25 Drug 2 Cases 10 30 0.333/1 = 0.333 Drug 2 Controls 30 30 Pooled Cases 610 230 2.65/1.91 = 1.39 Pooled Controls 105 55
- 17. Case-control analysis: Adjusted OR for duration of use of statins on survival Odds are relative to patients not on statins. Dotted line represents 1:1 odds 0.00 0.25 0.50 0.75 1.00 0−12 13−24 25−36 37−48 49−60 >60 Duration (months) AdjustedOddsratio PCD CPRD Qresearch
- 18. Summary — Statins study 1. Strikingly similar results in the two studies, despite different GP computer systems (Vision vs EMIS) 2. As expected, narrower confidence intervals due to larger study 3. Original study was well designed (Matching, appropriate analyses etc.) 4. Given the results, pooling of “all statins” is questionable
- 19. Does β-adrenoceptor blocker therapy improve cancer survival?(DIN-LINK) Objective To examine the effect of β-blocker treatment on cancer survival Design Survival analyses for 9 cancer types Setting 3462 cancer patients on β-blocker or other antihypertensive therapy Subjects Patients 40-85 with first cancer diagnosis between 1997 and 2006 Outcomes - Adjusted hazard ratios (+/- 95%CI) for all-cause mortality in each cancer type - Pooled hazard ratio (random effects)
- 20. Comparison of summary statistics Cohort size: CPRD 11316 DIN-LINK 3462 BPLM = Blood pressure lowering medicines Deaths Alive at 1 year Diagnosed 1997−2001 Diagnosed 2002−2006 0 25 50 75 100 Beta−blockersO therBPLM Beta−blockersO therBPLM Beta−blockersO therBPLM Beta−blockersO therBPLM Drug combination Percentageofpatients PCD CPRD DIN−LINK
- 21. Comparison of patient samples by cancer site BPLM = Blood pressure lowering medicines 0 10 20 30 40 breast colon lungoesophagus ovarian pancreas prostate renal stom ach Cancer site Percentageofpatients PCD CPRD DIN−LINK
- 22. Survival analyses: comparison of hazard ratios Cancer type breast colon lung oesophagus ovarian pancreas prostate renal stomach Overall Hazard ratio 1.09 (0.8, 1.49) 1 (0.77, 1.3) 1.12 (0.89, 1.41) 1.05 (0.69, 1.6) 1.14 (0.63, 2.06) 1.88 (1.09, 3.25) 1.54 (1.13, 2.09) 1.14 (0.52, 2.52) 1.44 (0.76, 2.74) 1.18 (1.04, 1.33) % weight 14.85 20.93 25.7 8.17 4.27 4.91 15.19 2.37 3.61 100 0 1 2 3 4 all Beta−blockers vs controls DIN−LINK Cancer type breast colon lung oesophagus ovarian pancreas prostate renal stomach Overall Hazard ratio 1.19 (1.03, 1.37) 0.85 (0.74, 0.97) 1.04 (0.91, 1.19) 1.27 (1.01, 1.59) 1.05 (0.74, 1.5) 0.94 (0.74, 1.21) 1.03 (0.92, 1.15) 0.46 (0.26, 0.83) 1.03 (0.78, 1.36) 1.01 (0.91, 1.13) % weight 14.87 15.11 15.36 10.7 6.32 9.76 16.27 2.89 8.71 100 0 1 2 3 4 all Beta−blockers vs controls CPRD
- 23. Summary — β-blocker study 1. Different individual cancer HR’s and overall conclusions 2. Important differences in some cohort statistics 3. Differences remain after correcting to give the same patient:practice 4. Differences remain after reducing the size of the CPRD study 5. Databases appear to be demographically similar (Carey et al. 2004)
- 24. Summary — β-blocker study 1. Different individual cancer HR’s and overall conclusions 2. Important differences in some cohort statistics 3. Differences remain after correcting to give the same patient:practice 4. Differences remain after reducing the size of the CPRD study 5. Databases appear to be demographically similar (Carey et al. 2004) WHY?
- 25. ”an experiment is reproducable until another laboratory tries to repeat it.” — Alexander Kohn
- 26. ”an experiment is reproducable until another laboratory tries to repeat it.” — Alexander Kohn • Artifact of differences in GP computer systems? DIN-LINK uses Torex/iSoft systems (See Kontopantelis et al 2013)
- 27. ”an experiment is reproducable until another laboratory tries to repeat it.” — Alexander Kohn • Artifact of differences in GP computer systems? DIN-LINK uses Torex/iSoft systems (See Kontopantelis et al 2013) • Analysis methods? – No matching - potential confounding – No clustering by practice – Limited control for covariates – Is meta-analysis the most appropriate method (Assumes independence)? • Data quality?
- 28. Conclusions / recommendations These replications add to the evidence that PCD results are valid
- 29. Conclusions / recommendations These replications add to the evidence that PCD results are valid • PCD Replication is hard!
- 30. Conclusions / recommendations These replications add to the evidence that PCD results are valid • PCD Replication is hard! – Methods details are inadequate for replication – Clinical codes not provided with the original article – Relies on active cooperation of authors of original studies – Even then, ambiguity can remain
- 31. Conclusions / recommendations These replications add to the evidence that PCD results are valid • PCD Replication is hard! – Methods details are inadequate for replication – Clinical codes not provided with the original article – Relies on active cooperation of authors of original studies – Even then, ambiguity can remain • Publish full methods (in online appendix?) • Publish full clinical code lists... ClinicalCodes.org
- 32. Thanks. . . Research team David Reeves Evan Kontopantelis Ivan Olier Darren Ashcroft Authors of the original studies Iain Carey (St. Georges University, London) Carol Coupland (University of Nottingham) Contact: david.springate@manchester.ac.uk