![Lessons learned from published economic evaluations in public
health | 11 November 2010
39
De Kok et al.: “In conclusion, many uncertainties still exist
about the effects of HPV vaccination on HPV-related diseases.
Our cost-effectiveness analysis shows that in the Netherlands,
a country with low cervical cancer incidence and mortality,
HPV vaccination is not cost-effective […].”](https://image.slidesharecdn.com/eupha2-lessonslearnedfrompublishedeconomicevaluationsbymathijsvandenberg-101118065842-phpapp02/85/Eupha-2-lessons-learned-from-published-economic-evaluations-by_mathijsvandenberg-39-320.jpg)
![Lessons learned from published economic evaluations in public
health | 11 November 2010
41
Postma: “I conclude that the conclusion by de Kok et al. is
misleading, should be re-visited and would probably better be
formulated as “In the Netherlands, HPV vaccination is likely to be
cost-effective if compared with screening alone”, […]”](https://image.slidesharecdn.com/eupha2-lessonslearnedfrompublishedeconomicevaluationsbymathijsvandenberg-101118065842-phpapp02/85/Eupha-2-lessons-learned-from-published-economic-evaluations-by_mathijsvandenberg-41-320.jpg)
![Lessons learned from published economic evaluations in public
health | 11 November 2010
47
● “The intervention had a gross incremental cost-effectiveness ratio of
AUD$ 3.70 (95% uncertainty interval (UI) $2.40, $7.70) per DALY.”
● “When the present value of potential savings in future health-care
costs was considered, the intervention was ‘dominant’ […].”
● “Restricting televised advertisements targeting children could
potentially be one of the most cost-effective population-based
obesity prevention interventions available to governments
today.The ICER was most sensitive to effectiveness assumptions.”
● “Limited evidence of the effectiveness of this intervention was a key
concern”](https://image.slidesharecdn.com/eupha2-lessonslearnedfrompublishedeconomicevaluationsbymathijsvandenberg-101118065842-phpapp02/85/Eupha-2-lessons-learned-from-published-economic-evaluations-by_mathijsvandenberg-47-320.jpg)
Eupha 2.lessons learned from published economic evaluations by_mathijsvandenberg
- 1. 1 11 November 2010 Lessons learned from published economic evaluations in public health Matthijs van den Berg Johan Polder Ardine de Wit
- 2. 2 Lessons learned from published economic evaluations in public health | 11 November 2010 Content 1. Introduction 2. RIVM project 3. Cost-effectiveness of public health interventions 4. Critical issues in public health economic evaluations
- 3. Lessons learned from published economic evaluations in public health | 11 November 2010 3 Cost-effectiveness is quite the thing ● Science – Exponential increase in cost-effectiveness studies – Both absolute and relative ● Practice ● Policy
- 4. Lessons learned from published economic evaluations in public health | 11 November 2010 4 Exponential increase cost-effectiveness studies 0 10000 20000 30000 40000 50000 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005
- 5. Lessons learned from published economic evaluations in public health | 11 November 2010 5 Exponential increase proportion c-e studies 0,0 0,1 0,1 0,2 0,2 0,3 0,3 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005
- 6. Lessons learned from published economic evaluations in public health | 11 November 2010 6 Cost-effectiveness is quite the thing ● Science ● Practice – Dutch manual for evidence-based guideline development: › Cost-effectiveness should be one of the ‘other considerations’ that should be taken into account before coming to recommendations (CBO, 2007) – Guidelines more frequently use cost-effectiveness considerations › Cardiovascular risk management ● Policy
- 7. Lessons learned from published economic evaluations in public health | 11 November 2010 7 Dutch guideline ‘Cardiovascular risk management’ ● Guideline: – CVD 10 year mortality risk > 5%: lifestyle advice – CVD 10 year mortality risk >10%: pharmacological treatment ● Recommendations are based on analyses of cost-effectiveness and budget impact ● The threshold for cost-effectiveness is set on € 20.000 per QALY gained, which corresponds with the value of ƒ 40.000 in the 1998- guideline, corrected for inflation
- 8. Lessons learned from published economic evaluations in public health | 11 November 2010 8 Cost-effectiveness is quite the thing ● Science ● Practice ● Policy – Dutch policy document on public health: ‘Being healthy, and staying healthy; a vision of health and prevention’ – Health care insurance board: judgement framework for reimbursement decisions on the basic health care package
- 9. Lessons learned from published economic evaluations in public health | 11 November 2010 9 ● One of the main principles in public health policy making – “Forms of preventive intervention that do not work will be stopped,… – and promising forms of intervention will be investigated to establish how (cost-)effective they are.... – Forms of intervention that are known to be (cost-)effective should, in principle, be funded through the appropriate insurance systems, subject to the usual budgetary constraints.”
- 10. Lessons learned from published economic evaluations in public health | 11 November 2010 10 ● Dutch Health Insurance Board uses a judgement framework for which interventions to cover, consisting of 4 criteria: – Necessity – Effectiveness – Cost-effectiveness – Feasibility ● Health Council of the Netherlands uses frameworks for decisions on new vaccinations and screenings – Cost-effectiveness is a main assessment criterion in both frameworks
- 11. Lessons learned from published economic evaluations in public health | 11 November 2010 11 Take-home message ● Cost-effectiveness is hot, especially in public health, but don’t take the result of a cost-effectiveness analysis (i.e. an ICER) for granted ● Read cost-effectiveness analyses of preventive interventions very carefully and ask yourself questions like: – Are the assumptions on effectiveness well-founded and realistic? – Which costs are included and which are ignored? – What are critical choices in the model that was used?
- 12. Lessons learned from published economic evaluations in public health | 11 November 2010 12 RIVM project ‘Cost-effectiveness of prevention’ ● Aim of the project – To identify potentially cost-effective preventive interventions that have not yet been systematically implemented in the Netherlands. – i.e. to find opportunities for public health policy ● Basic idea of the project: – Systematic and frequent literature search – Read and appraise the cost-effectiveness analyses – Write a report or a paper
- 13. Lessons learned from published economic evaluations in public health | 11 November 2010 13
- 14. Lessons learned from published economic evaluations in public health | 11 November 2010 14
- 15. Lessons learned from published economic evaluations in public health | 11 November 2010 15 Number and type of prevention-CEAs ● In 2008 we found over 230 papers studying the cost-effectiveness of a preventive intervention or program ● About half of the included studies used QALY’s of DALY’s as outcome measure
- 16. Lessons learned from published economic evaluations in public health | 11 November 2010 16 What diseases do prevention-CEAs focus on? – Infectious diseases: 32% – Cancers: 21% – Cardiovascular diseases: 10% – Mental- and behaviour diseases: 7% – Musculoskeletal system: 7% – Rest 23% ● Vaccination gets disproportionally much attention in cost- effectiveness analyses relative to the burden of infectious diseases ● Mental- and behavioural diseases cause large burden of disease, while they were the subject of only 7% of included studies
- 17. Lessons learned from published economic evaluations in public health | 11 November 2010 17 How cost-effective were the studied interventions? >80% lower than 50,000/(QA)LY Median: €12,000/(QA)LY ● What do you conclude from this chart?
- 18. Lessons learned from published economic evaluations in public health | 11 November 2010 18 How cost-effective were the studied interventions? ● Almost all preventive interventions have favourable cost- effectiveness ratios – Is prevention always cost-effective? – Is there a publication bias? – Do researchers work up to favourable ICERs?
- 19. Lessons learned from published economic evaluations in public health | 11 November 2010 19 Some critical comments ● Based on observations in economic evaluations of public health interventions ● Three categories – Costs (numerator of the ICER) – Effects (denominator of the ICER) – Models
- 20. Lessons learned from published economic evaluations in public health | 11 November 2010 20 Costs ● Which costs are included? – Health care perspective (only health care costs and health benefits) or societal perspective (all costs and all benefits) – The societal perspective is preferred – Public health measures have impact outside the healthcare sector – Most CEA’s on prevention use the health care perspective, ignoring very relevant cost categories
- 21. Lessons learned from published economic evaluations in public health | 11 November 2010 21 Effects ● Is the intervention scenario realistic? – What is the reach of the intervention? What participation rate is assumed? – Case: colorectal cancer screening using colonoscopy
- 22. Lessons learned from published economic evaluations in public health | 11 November 2010 22 •Unpleasant bowel preparation •Uncomfortable proceduce •Risk of bowel perforation
- 23. Lessons learned from published economic evaluations in public health | 11 November 2010 23 Participation rate ● Is the intervention scenario realistic? – What is the reach of the intervention? What participation rate is assumed? – Case: colorectal cancer screening using colonoscopy – How many people would participate in such a screening?
- 24. Lessons learned from published economic evaluations in public health | 11 November 2010 24
- 25. Lessons learned from published economic evaluations in public health | 11 November 2010 25 Participation rate ● Systematic review of cost-effectiveness analyses of colonoscopy screening – Model assumptions on uptake: ~85% – Real life data for participation: ~30% ● Conclusion: – In many CEAs model assumptions on screening uptake were more positive than real life data suggest
- 26. Lessons learned from published economic evaluations in public health | 11 November 2010 26 Adherence
- 27. Lessons learned from published economic evaluations in public health | 11 November 2010 27 Adherence ● Systematic review of medication CEAs found that many studies used adherence rates from clinical trials which notoriously overestimate adherence found in the general population ● However in a small subset of studies that varied adherence in sensitivity analyses, the impact on the cost-effectiveness ratio was substantial ● With policymakers increasingly turning to these studies for guidance, failure to account for nonadherence may lead to suboptimal resource allocation strategies
- 28. Lessons learned from published economic evaluations in public health | 11 November 2010 28 Uncertainty in effectiveness
- 29. Lessons learned from published economic evaluations in public health | 11 November 2010 29 Uncertainty in effectiveness ● “The aim of this paper was to determine the cost-effectiveness of a Dutch school-based smoking education program. ● The incremental cost-effectiveness ratio of the school program was estimated at €19,900 per quality adjusted life year gained. ● Main problem in estimating the cost-effectiveness was the lack of proper effectiveness data on daily smokers among adolescents.”
- 30. Lessons learned from published economic evaluations in public health | 11 November 2010 30
- 31. Lessons learned from published economic evaluations in public health | 11 November 2010 31 Long term effects
- 32. Lessons learned from published economic evaluations in public health | 11 November 2010 32 Long term effects
- 33. Lessons learned from published economic evaluations in public health | 11 November 2010 33
- 34. Lessons learned from published economic evaluations in public health | 11 November 2010 34 ● “The range of effectiveness of the vaccine was considered between 30% en 90% with a positive baseline at 80%.” ● “The costs of the vaccine was assumed to be $450 for three doses and ranged from $300 to $2000.” ● “Vaccination was found to be potentially cost-effective with an ICER of -$2384 relative to standard treatment.” › However, such a vaccine is not available at the moment › This is a purely hypothetical excercise
- 35. Lessons learned from published economic evaluations in public health | 11 November 2010 35 Models ●Time horizon –Many modelling studies on the cost-effectiveness of preventive interventions have very long time horizons (E.g. 80 yrs, 100 yrs, lifetime) –Necessary to capture all health benefits (e.g. stop smoking interventions and the prevention of lung cancer) –Shorter time horizons would make the ICERs of many preventive interventions very unfavourable
- 36. Lessons learned from published economic evaluations in public health | 11 November 2010 36 Models
- 37. Lessons learned from published economic evaluations in public health | 11 November 2010 37 Models ●Discount rates – Most international CEAs use same discount rate for both costs and effects (e.g. 4% or even 6%) – Dutch guidelines prescribe different discount rates for costs and effects (4% and 1.5%) – This has a large influence on the ICER, making prevention a lot more cost-effective
- 38. Lessons learned from published economic evaluations in public health | 11 November 2010 38 ●Case: cost-effectiveness of HPV vaccination – Dozens of recently published CEAs – Including several Dutch model studies – Most of these conclude favourable cost-effectiveness – However, de Kok et al., 2009 does not
- 39. Lessons learned from published economic evaluations in public health | 11 November 2010 39 De Kok et al.: “In conclusion, many uncertainties still exist about the effects of HPV vaccination on HPV-related diseases. Our cost-effectiveness analysis shows that in the Netherlands, a country with low cervical cancer incidence and mortality, HPV vaccination is not cost-effective […].”
- 40. Lessons learned from published economic evaluations in public health | 11 November 2010 40 Coupé et al.: “In a sensitivity analysis, de Kok et al. reported an ICER of € 19 700 per QALY at discount rates of 4% for costs and 1.5% for effects. The difference in reported ICERs between the two studies can thus be largely explained by the discount rates that were used.
- 41. Lessons learned from published economic evaluations in public health | 11 November 2010 41 Postma: “I conclude that the conclusion by de Kok et al. is misleading, should be re-visited and would probably better be formulated as “In the Netherlands, HPV vaccination is likely to be cost-effective if compared with screening alone”, […]”
- 42. Lessons learned from published economic evaluations in public health | 11 November 2010 42 ●Moreover – Assumptions concerning participation rate in HPV vaccination were about 85% – While the real participation rate in the Netherlands appeared to be about 50% – In what way would that influence the cost-effectiveness ratio?
- 43. Lessons learned from published economic evaluations in public health | 11 November 2010 43 Another example: rotavirus vaccination
- 44. Lessons learned from published economic evaluations in public health | 11 November 2010 44
- 45. Lessons learned from published economic evaluations in public health | 11 November 2010 45
- 46. Lessons learned from published economic evaluations in public health | 11 November 2010 46
- 47. Lessons learned from published economic evaluations in public health | 11 November 2010 47 ● “The intervention had a gross incremental cost-effectiveness ratio of AUD$ 3.70 (95% uncertainty interval (UI) $2.40, $7.70) per DALY.” ● “When the present value of potential savings in future health-care costs was considered, the intervention was ‘dominant’ […].” ● “Restricting televised advertisements targeting children could potentially be one of the most cost-effective population-based obesity prevention interventions available to governments today.The ICER was most sensitive to effectiveness assumptions.” ● “Limited evidence of the effectiveness of this intervention was a key concern”
- 48. Lessons learned from published economic evaluations in public health | 11 November 2010 48 ● “The most relevant study to assess the effectiveness of a reduction in advertising was a randomized controlled trial that compared the impact of exposure to different food advertisements on food and beverage selections in 288 5- to 8-year old children on holiday camp for a period of 2 weeks in Quebec in 1982. ● The children were exposed to sweet commercials, no commercials, fruit commercials or nutritional public service announcements. Children who viewed sweet commercials chose (and ate) significantly more sweets over fruit as snacks compared with children in the other three groups (75 versus 67%, 64 and 65%, respectively).” ● “Whether te reduction in BMI would be maintained over the lifetime of the child is unknown and difficult to predict.”
- 49. Lessons learned from published economic evaluations in public health | 11 November 2010 49 Conclusions ●Don’t take the result of a cost-effectiveness analysis (i.e. an ICER) for granted ●Please read cost-effectiveness analyses of preventive interventions very carefully – Are the effect assumptions realistic? – Are all relevant costs included? – What are critical model choices?