Reviewing for model parameters workshop, HTAi 2014
Download as PPT, PDF1 like307 views
This document summarizes a workshop on reviewing model parameters for health technology assessments. It identifies six key themes from focus groups: 1) Current practice involves iterative discussions and modeling from the beginning. 2) What constitutes "enough" information is unclear. 3) Information needs can arise at different stages. 4) An ideal process involves early team involvement in problem structuring and focused searching. 5) Early structuring of important parameters is important. 6) Further research is needed on focused searching, quality assessment, and validating estimates. The document concludes more guidance is needed to make the review process transparent, timely, and rigorous.
Reviewing for model parameters workshop, HTAi 2014
- 1. Reviewing for model parameters workshop 7 February 2011 Eva Kaltenthaler Paul Tappenden Suzy Paisley
- 2. DSU Technical Brief • Practical guidance on reviewing for model parameters • March 2011
- 3. Background • HTA work for NICE/NETSCC • “For all parameters (including effectiveness, valuation of HRQL and costs) a systematic consideration of possible data sources is required” (NICE, 2008) • Perspective as a systematic reviewer
- 4. TARs • Focused question (PICO)-clinical effectiveness review • Decision problem-cost effectiveness model • Additional information needed for model parameters
- 6. Points to consider • Rapid timelines • Transparent • Reproducible • Systematically done • Attempts to reduce bias • Not just a series of rapid reviews
- 7. Six themes from Focus Groups • Current practice: discussions from the beginning, iterations • Adequate information: how much is enough? • Timing: need for information on parameters can happen at different stages
- 8. Themes (continued) • Ideal practice: team involved in problem structuring, “tagging of references”, focussed searching • Problem structuring: what parameters are likely to be important • Areas for further research: focussed searches, quality and selection decisions
- 9. Conclusions • Reviewing, searching and modelling were seen as integrated tasks • Whole team should be involved in structuring the decision problem • Good communication is important • Assessments of the quality and relevance of information are important
- 10. Conclusions • This preliminary investigation highlights numerous concerns and potential deficiencies in the process of identifying, selecting and using evidence to inform models.
- 11. Future research • Training is needed for focussed searching, problem structuring, quality assessment and the validation of parameter estimates. • Guidance is required to ensure that such research activity is transparent, timely and rigorous.
- 12. DSU Technical Brief 1. What methods for problem structuring can be recommended? 2. How should evidence be systematically identified to inform model parameter estimates?
- 13. 3. What guidance can be provided about the methods to use for reviewing model parameter data in a systematic fashion? 4. What recommendations concerning reporting can be made?
Editor's Notes
- #3: Experts in the field Range of perspectives Consensus on practical advice and best practice Different approaches
- #4: First TAR-clinical effectiveness review and then more requests for info. How to do it? Sometimes very little time. NICE methods guide A few points- mainly dealing with TARs work here although many of the issues are applicable to other areas Coming at it from the viewpoint of a systematic reviewer. These issues are important to reviewers, modellers and information specialists which is why you have all been invited.
- #5: The beginning Most TARs start with a scope or focussed question defined in terms of PICO. This is connected to the actual decision problem the TAR is addressing which is the foundation for the model. More information is needed to populate the CE model than that found in the review.
- #7: Important issues to consider when reviewing for these parameters, may be reviewer or modeller depending on various things such as time, communication, NICE methods guide very vague on this
- #8: We decided to hold a couple of internal focus groups. Some teams work better than others, sometimes there is conflict, things get missed. What works and what doesn’t. What do we know and where do we need more research? 1. Team work, good communication –it doesn’t always happen 2. How much is enough, not just where you found the information but why you made certain decisions. 3. Team need to know from the beginning about what to look out for, discussions about what might become important, a little contingency time for unexpected parameters
- #9: 4. Good communication was deemed to be essential with more time spent on the most important information requirements. 5. Problem structuring: important to look at the whole structure of the model, not just the parameters in it. 6. Assessments of the quality and relevance of information were also considered important. Future research needs include training for focussed searching, problem structuring, quality assessment and the validation of parameter estimates.
- #13: This will include the identification, specification and prioritisation of relevant parameters. This will include recommendations on relevant sources and on focussed searching.
- #14: How does this differ from conventional reviews of clinical effectiveness? This will include recommendations for critical appraisal and rapid review methods. This will include the reporting of review methods and of decisions and judgments regarding the identification and selection of parameters.