Effective ex post Evaluation: Purpose and Challenges
1 like1,839 views
The presentation discusses the importance of ex post evaluations in regulatory performance, highlighting their role in ensuring regulations remain effective, cost-effective, and aligned with policy goals. It identifies various costs associated with regulations and proposes strategies for effective evaluations, including programmed and ad hoc reviews. Challenges such as resource scarcity and political inertia are noted, alongside suggestions for improving Australia's regulatory evaluation practices.
Effective ex post Evaluation: Purpose and Challenges
- 1. Effective ex-post evaluation: purpose and challenges Gary Banks Presentation to OECD’s 9th Expert Meeting on Measuring Regulatory Performance June 2017 Lisbon, Portugal
- 2. OECD Council Recommendation #5 “Conduct systematic reviews of regulation against policy goals, including costs and benefits, to ensure that regulations -- remain up to date, cost effective and consistent, and -- deliver intended policy objectives” 2
- 3. Ex post evaluation: why needed? • All regulations are ‘experiments’ ─ Many will not have been done well ─ Others will have passed their ‘use by date’ • The ‘stock’ of regulation is much greater than the flow and gains are potentially large • Evaluations can build trust and help sustain political support • Ex post reviews can also provide learnings for future regulatory actions 3
- 4. 4 There are multiple costs of regulation Benefits foregone if regulation is ineffective other perverse effects other ‘non -market’ distortions Fees and charges a Economic distortions dead weight losses lower investment lower innovation Substantive compliance costs investments in systems training higher cost of investment Administrative costs to business paper work time reporting time Administration cost to regulators Benefitsneededtojustifycosts Compliance costs ‘Distortion’ costs Costs to government Costs to business b Costs to community Coststocommunity
- 5. 5 Ex post reviews complete the ‘regulatory cycle’ Stage II–Establishment Design to include embedded reviews Development of regulator management strategies Stage IV- Review Programmed reviews o Sunsetting o Embedded o PIRS Ad hoc reviews o In-depth o Specific benchmarking o Public stocktakes Stage I- Decision RIS triage o Identify need for embedded reviews o Set sunset flags Stock-flow linkage rules Stage III - Administration Regulator management strategies Monitoring review requirements Lessons from regulators on what works Lessons from expost evaluation
- 6. What is an effective system? • Comprehensive coverage over time − including related instruments • Targeted and proportionate − where most potential for net gains • Timeliness of reviews • Effective consultation mechanisms • Independence where appropriate • Institutionalised oversight − Monitoring performance of the overall system 6
- 7. 7 Approaches to ex post evaluation Programmed reviews Ad hoc reviews Ongoing ‘management’ •Sunsetting •Embedded in statute •Post implementation reviews – process failure – catch-all •Public stocktakes •‘Principles-based’ reviews •Benchmarking •‘In-depth’ reviews •Stock-flow linkages – Budgets – ‘In-Out’ / ‘Offsets’ •Red tape reduction targets
- 8. Key questions to be answered … • Appropriate? (Still a valid rationale?) • Effective? (Achieved the intended outcome?) • Efficient? (Unnecessary costs or unintended impacts?) • A better alternative? 8
- 9. Fewer countries have formal requirements Source: 2015 Regulatory Policy Outlook 9 28 24 28 ConsultationRIA Ex-post evaluation of existing regulation 21 24 20 3433 29 35
- 10. How do OECD countries measure up? 10 10 Source: OECD, Regulatory Policy Outlook 2015 http://www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm
- 11. Ex post reviews in Australia: a ‘mixed bag’? According to a Productivity Commission review, improvements were needed in the following areas: • embedding evaluations in regulatory design • proportionality/depth of reviews • prioritisation and monitoring • timeliness (PC 2011) 11 11
- 12. Some obstacles to better practice • ‘Technical’ −(Highly) skilled people are scarce −Data are often poor • ‘Political’ −Greater focus on the new than the old −Perceived risks from a poor ‘score’ −Beneficiaries may oppose scrutiny −Administrative ‘inertia’ is ever present! 12
- 13. Ways forward? Insights from Australia • Building ex post evaluation into a regulatory regime from the outset has advantages • ‘Automatic’ triggers can be useful • Attention to timing can make a big difference • Skill development is fundamental • Need avoid administrative ‘overload’ • Creating oversight arrangements helps 13
- 14. Effective ex-post evaluation: purpose and challenges Gary Banks Presentation to OECD’s 9th Expert Meeting on Measuring Regulatory Performance June 2017 Lisbon, Portugal