Quantifying Regulatory Efficacy
0 likes2,724 views
The document discusses the importance of evaluating the efficacy of regulations related to chemical management, highlighting the need for feedback to inform both existing and future regulations. It outlines the challenges associated with ex-ante and ex-post analyses, including methodological issues and a lack of incentives for robust evaluation. The document concludes that enhancing evaluation methods and addressing incentive challenges are critical for improving regulatory outcomes and policymaking.
Quantifying Regulatory Efficacy
- 1. Quantifying Regulatory Efficacy OECD Workshop on Socioeconomic Impact Assessment of Chemicals Management Susan E. Dudley, Director Helsinki, Finland 6-8 July 2016
- 2. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Agenda • Why evaluate efficacy of regulation? • How does evaluation fit in the regulatory process? • Why is evaluation so challenging? • Methods for better regulatory evaluation • Improving incentives for robust evaluation • Discussion 2
- 3. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Why evaluate? • Feedback is important • Systems mindset • Inform existing regulations – Delivering expected results? – Revise or rescind? • Inform future regulations/policies – Improve ex ante analysis – Understand causal relationships 3
- 4. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Scientific method 4 Observation Hypothesis Experiment Results Revise or Support
- 5. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Regulatory process • Regulatory impact assessment (ex ante) – Hypothesize causal links between action & outcome – Assess risks (models, assumptions) – Identify alternative actions to manage risks – Estimate benefits, costs & distributional effects • Public engagement (ex ante) • Implementation • Ex post evaluation? 5
- 6. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Challenges Ex-ante analysis • Well-established principles & procedures • Challenging methodologically – Assumptions and models – Hypothesize outcomes Ex-post analysis • Procedures less well- established • Challenging methodologically – Counterfactual? – Opportunity costs? – Measuring actual reduction in risks • Incentives lacking 6
- 7. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Responding to Methodological Challenges • Plan for retrospective review at outset • Design rules to allow natural experiments • Lay out theory of change • Determine proper scope • Measure causal relationship 7
- 8. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Plan for review at the outset • Problem to be solved • Criteria for measuring success • Causal logic model • Time frame 8
- 9. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Design to enhance learning • Randomized controlled experiment • Variation facilitates natural experiments – Pilots or trials – Different compliance thresholds or timing – Differences across jurisdictions 9
- 10. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Illustrative Causal Model Policy Design • Problem identification, • Risk assessment • Impact analysis Policy Implementation • Enforcement Behavioral Change • Compliance costs • Emission reductions • Substitution Outputs • Concentration changes • Exposure changes • Opportunity costs • Co-benefits & costs Environmental & Health Outcomes • Risk reduction • Benefits • Unintended effects 10
- 11. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Scope • Specific regulation vs. program-wide • Outputs vs. outcomes • Quantified benefits & costs • Other factors – Innovation – Flexibility – Distribution 11
- 12. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Measure causal relationship • Quasi-experimental approaches – Over time – Across jurisdictions • Statistical tests – Is X informative about Y? – Does X precede Y? – Does ∆X explain ∆Y? – What do non-parametric methods show? – Effect of X w/ other variables held constant 12
- 13. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Responding to Incentive Challenges • Condition new regulation on ex post learning • Institutionalize independent review • Change default rules • Reallocate resources 13
- 14. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Conclusions • Evaluation & feedback important – Identify underperforming (or overly costly) rules – Improve future policy • Modify ex ante assumptions • Calibrate risk assessments • Improve causal predictions of health benefits & other regulatory effects • Solutions must address methods & incentives 14
- 15. Susan E. Dudley @RegStudies RegulatoryStudies.gwu.edu Contact: Susan E. Dudley @RegStudies RegulatoryStudies@gwu.edu Regulatorystudies.gwu.edu