Annual Results and Impact Evaluation Workshop for RBF - Day One - Verification of Results Group Work
0 likes247 views
This document discusses verification processes for results-based financing (RBF) programs. It presents a pathway tool to examine quantity verification error rates, patient tracing error rates, and counter-verification error rates. The tool analyzes patterns at the facility level, indicator level, and verifier level. It also explores potential causes of errors, such as insufficient training, complex compliance criteria, or incentive amounts that encourage gaming. Addressing high error rates may involve investigating their underlying causes through facility visits, increasing sanctions, providing additional training or support, or adjusting the verification methodology. The overall goal is to expand knowledge of verification practices and improve the accuracy of operational data from RBF programs.
Annual Results and Impact Evaluation Workshop for RBF - Day One - Verification of Results Group Work
- 1. Verification of Results Group Work Petra Vergeer | Senior Health Specialist
- 2. Overview 1 • Operational data from RBF programs is of limited use unless it is accurate. • A cross-case analysis was carried out to expand knowledge about verification processes and practices. o Afghanistan,Argentina,Burundi, and Rwanda • Introduction of verification “pathway tool” o Initial consultation to obtain feedback to further develop tool
- 4. Examine quantity verification error rates 1 Facility-level patterns Indicator-level patterns Generalized across all contracted parties Localized to specific facilities (by geographic area) Localized to specific types of facilities Indicators with complex compliance criteria Indicators that are rewarded more frequently and/or have higher patient volume Indicators rewarded at a higher level Examine quantity verification error rates Activities to explore Activities to explore
- 5. Examine patient tracing error rates Examine patient tracing error rates Facility-level patterns Indicator-level patterns Generalized across all contracted parties Localized to specific facilities (by geographic area) Localized to specific types of facilities Indicators with complex compliance criteria Indicators that are rewarded more frequently and/or have higher patient volume Indicators rewarded at a higher level Activities to explore Activities to explore
- 6. Examine counter-verification error rates Verifier-level patterns Indicator-level patterns Generalized across all verified parties Localized to a specific verifier Localized to a specific type of facility Examine quantity and quality counter- verification error rates (Assume data are at level of contracted party.) Indicators with complex compliance criteria Indicators that are rewarded more frequently and/or have higher patient volume Indicators rewarded at a higher level Activities to explore Activities to explore
- 7. Factors influencing verification: a conceptual framework Context Verification Characteristics Impacton accuracy, cost, sustainability RBF Characteristics RATIONALE FOR RBF CONTRACT TYPE USE OF RBF RESULTS Improvinghealthoutcomes/HSS Relational Payment,improvingperformance Financial accountability/Costcontrol Classic Transparency, Namingand Shaming Monthly Annual Yes Large Whole universe Risk-basedapproach Internal Verification Results and Their Use FREQUENCY ALLOWABLE ERROR MARGIN SAMPLE SIZE INSTITUTIONAL SETUP ADVANCE WARNING No Small Third party Learning, Error correction Cost recovery, Sanction PAYMENT FREQUENCY Monthly Annual POLITICAL ENVIRONMENT GOVERNANCE CULTURE
- 8. Potential causes of generalized error rates: verification • Knowledge and tools o Insufficient training for providers, data entry clerks o Data collection tools insufficiently clear o Explore validity of verification methods • Allowable error margin o Assess error margin • Advance warning • Institutionalsetup o There is a conflict of interest betweenthe verifier and the contracted party • Sample size o Data are “noisy” • Frequency o Verification/counter-verification occurs too infrequently to deter gaming • Verification results and use o Incentives and sanctions are not sufficient to mitigate mis-reporting Question: What are the causes in your country? Back to main slide
- 9. Potential causes of generalized error rates: patient tracing • Knowledge and tools o Insufficient training for providers, data entry clerks o Data collection tools insufficiently clear o Explore validity of verification methods • Allowable error margin o Assess error margin • Advance warning • Institutionalsetup o There is a conflict of interest betweenthe verifier and the contracted party • Sample size o Data are “noisy” • Frequency o Verification/counter-verification occurs too infrequently to deter gaming • Verification results and use o Incentives and sanctions are not sufficient to mitigate mis-reporting Question: What are the causes in your country? Back to main slide
- 10. Potential causes of generalized error rates: counter-verification • Knowledge and tools o Insufficient training for providers, data entry clerks o Data collection tools insufficiently clear o Explore validity of verification methods • Allowable error margin o Assess error margin • Advance warning • Institutionalsetup o There is a conflict of interest betweenthe verifier and the contracted party • Sample size o Data are “noisy” • Frequency o Verification/counter-verification occurs too infrequently to deter gaming • Verification results and use o Incentives and sanctions are not sufficient to mitigate mis-reporting Question: What are the causes in your country? Back to main slide
- 11. Potential causes of error rates localized to a specific geographic area: verification • Knowledge and tools o Insufficient support for contracted parties • Allowable error margin • Advance warning • Institutional setup • Sample size • Frequency o Verification occurs too infrequentlyto deter gaming • Verification results and use Question: What are the causes in your country? Back to main slide
- 12. Potential causes of error rates localized to a specific geographic area: patient tracing • Knowledge and tools o Insufficient support for contracted parties • Allowable error margin • Advance warning • Institutional setup • Sample size • Frequency o Verification occurs too infrequentlyto deter gaming • Verification results and use Question: What are the causes in your country? Back to main slide
- 13. Potential causes of error rates localized to a specific verifier: counter-verification • Knowledge and tools o Insufficient support for verifiers • Allowable error margin • Advance warning • Institutional setup • Sample size • Frequency o Verification occurs too infrequentlyto deter gaming • Verification results and use Question: What are the causes in your country? Back to main slide
- 14. Potential causes of error rates localized to a specific type of facility: verification • Knowledge and tools • Allowable error margin o Explore whether error margin is appropriate • Advance warning • Institutional setup • Sample size o Biased data due to insufficient sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 15. Potential causes of error rates localized to a specific type of facility: patient tracing • Knowledge and tools • Allowable error margin o Explore whether error margin is appropriate • Advance warning • Institutional setup • Sample size o Biased data due to insufficient sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 16. Potential causes of error rates localized to a specific type of facility: counter-verification • Knowledge and tools • Allowable error margin o Explore whether error margin is appropriate • Advance warning • Institutional setup • Sample size o Biased data due to insufficient sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 17. Potential causes of high error rates among indicators with complex compliance criteria: verification • Knowledge and tools o Confusing data collection tools o Inadequate training for providers and data entry clerks • Allowable error margin o Indicator too complex for accurate reporting • Advance warning • Institutional setup • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 18. Potential causes of high error rates among indicators with complex compliance criteria: patient tracing • Knowledge and tools o Confusing data collection tools o Inadequate training for providers and data entry clerks • Allowable error margin o Indicator too complex for accurate reporting • Advance warning • Institutional setup • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 19. Potential causes of high error rates among indicators with complex compliance criteria: counter-verification • Knowledge and tools o Confusing data collection tools o Inadequate training for providers and data entry clerks • Allowable error margin o Indicator too complex for accurate reporting • Advance warning • Institutional setup • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 20. Potential causes of high error rates among indicators that are rewarded more frequently or have higher patient volume: verification • Knowledge and tools • Allowable error margin • Advance warning • Institutional setup • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 21. Potential causes of high error rates among indicators that are rewarded more frequently or have higher patient volume: patient tracing • Knowledge and tools • Allowable error margin • Advance warning • Institutional setup • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 22. Potential causes of high error rates among indicators that are rewarded more frequently or have higher patient volume: counter-verification • Knowledge and tools • Allowable error margin • Advance warning • Institutional setup • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 23. Potential causes of high error rates among indicators that are rewarded at a higher level: verification • Knowledge and tools • Allowable error margin • Advance warning • Institutional setup o Incentive amount encourages gaming • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 24. Potential causes of high error rates among indicators that are rewarded at a higher level: patient tracing • Knowledge and tools • Allowable error margin • Advance warning • Institutional setup o Incentive amount encourages gaming • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 25. Potential causes of high error rates among indicators that are rewarded at a higher level: counter-verification • Knowledge and tools • Allowable error margin • Advance warning • Institutional setup o Incentive amount encourages gaming • Sample size • Frequency • Verification results and use Question: What are the causes in your country? Back to main slide
- 26. Examples of options to address high error rates • Visit small number of facilities to investigate underlying causes of error rate patterns • Increase sanction amount/proportion • Conduct additional trainings for providers, data entry clerks • Provide additional support on an ongoing basis for providers, data entry clerks
- 27. Thank You