Prevention of corruption, developing indicators and measuring achievements
Download as PPT, PDF3 likes2,354 views
The document outlines the development of indicators and methodologies for measuring achievements in preventing corruption, focusing on participatory processes and the RBM life-cycle approach. It emphasizes the importance of actionable indicators that are sensitive to gender and poverty, alongside the need for effective assessments and stakeholder engagement. The document also discusses various types of indicators, data collection methods, and the significance of both perception-based and experience-based data in understanding corruption.
![Strategy Goal 2: A decrease in corruption in the judiciary ( perceptions) What questions could you ask of citizens to help monitor this goal? Do you think corruption in the judiciary has decreased over the past x years? In your opinion, do you think there is a higher, equal, or lower level of corruption in the judiciary as compared to [insert another key institution]? In your opinion, do you think there is a higher, equal, or lower level of corruption in the judiciary as compared to [insert neighboring country]?](https://image.slidesharecdn.com/preventionofcorruptiondevelopingindicatorsandmeasuringachievements-111128025830-phpapp01/85/Prevention-of-corruption-developing-indicators-and-measuring-achievements-44-320.jpg)
![Thank you! Asmara Achcar Regional Specialist, Governance Assessments [email_address]](https://image.slidesharecdn.com/preventionofcorruptiondevelopingindicatorsandmeasuringachievements-111128025830-phpapp01/85/Prevention-of-corruption-developing-indicators-and-measuring-achievements-49-320.jpg)
Prevention of corruption, developing indicators and measuring achievements
- 1. Developing Indicators and Measuring Achievements AdvancedTraining on Prevention of Corruption Techniques &Methodologies 10-11 November 2011 Asmara Achcar
- 2. Applying the tools to the RBM life-cycle approach? IMPACT OUTPUT Planning Evaluation Monitoring Capacity Capacity Capacity Capacity Capacity Participatory process
- 3. Overview of presentation Designing indicators Measuring/Monitoring your indicators Potential models/examples
- 4. Overview of presentation Designing indicators Basic steps Actionable indicators Different indicator approaches Measuring/Monitoring your indicators Potential models/examples
- 5. Key steps in conducting an effective assessment Identify national institution or civil society organisation for hosting initiative Conduct multi-stakeholder dialogue on governance priorities Decide on assessment framework Decide on indicators Decide on sampling Analyse results Disseminate results Conduct multi-stakeholder consultation Develop policy recommendations Conduct policy reform Institutionalize and repeat at regular intervals Identify key stakeholders Decide how to collect data Establish a steering committee Raise funds Decide who will do the research Select type of assessment
- 6. 10 features of an effective assessment development Strategy is aligned to national policy priorities and processes Strategy is country contextualized Methodology is rigorous Selection of indicators is transparent and participatory Results are stored in a public national database Indicators are pro-poor and gender-sensitive Capacity of national stakeholders is developed Assessment is cost-effective and timely Results are widely disseminated Assessment is repeated
- 7. Why participatory adaptation? Technical benefits Customized Actionable Political benefits Buy-in Credibility Legitimacy and public trust Consensus-building and political will Efficiency benefits (usage) Adapted to needs Integrated in planning Sustainable Increase impact of assessment
- 8. Participation at 2 levels Through process : Participation in design Participatory monitoring Through methodology : Inclusive data sources Gender-sensitive and pro-poor governance indicators
- 9. Stakeholder Analysis Identifying the key stakeholders and their interests in reform (positive or negative) Assessing the influence and importance of each stakeholder Identifying measuring needs of stakeholders (basis for engagement)
- 10. Significant influence Some influence Little influence No influence Significantly interested in reform Some interest Little interest No interest
- 11. Significant influence Some influence Little influence No influence Significantly interested in reform Ministry of Local Government state commission for the prevention of corruption citizens NGO 2 NGO 3 The poor Women Some interest President of municipal councils ministry of transport NGO 1 state audit office bureau of public procurement Media 2 NGO 4 Little interest Mayors Business community Media 1 ministry of environmental protection and urban planning No interest Chief of administration Local public servants
- 12. THE RBM LIFE-CYCLE APPROACH
- 13. Applying the tools to the RBM life-cycle approach? IMPACT OUTPUT Planning Evaluation Monitoring Capacity Capacity Capacity Capacity Capacity Participatory process
- 14. Overview of presentation Designing indicators Basic steps Actionable indicators Different indicator approaches Measuring/Monitoring your indicators Potential models/examples
- 15. Which, if either, of these indicators are actionable? 1. How often do firms make extra payments to influence the content of new legislation? 2. How many judges and magistrates do you think are involved in corruption? How would you make them more ‘actionable’?
- 16. Activity: two examples Non-actionable 1. How often do firms make extra payments to influence the content of new legislation? 2. How many judges and magistrates do you think are involved in corruption? Actionable In practice, are the regulations restricting post-government private sector employment for national legislators effective? In practice, are the requirements for the independent auditing of the asset disclosure forms of members of the national legislature fulfilled? In practice, when necessary, does the judicial disciplinary agency initiates investigations? In practice, when necessary, does the judicial disciplinary agency impose penalties on offenders?
- 17. Actionable or ‘action-worthy’? Actionable does not mean action-worthy e.g. Measuring whether a country has an independent anti-corruption commission when there is no guarantee it will reduce corruption e.g. Measuring the speed of judicial proceedings when not clear that increasing the speed will ensure that justice is done There is a risk of measuring things because they are easily measurable
- 18. Determining what is an ‘action-worthy indicator’ Is it measuring interventions that are truly ‘beneficial’? Must be appropriate for country context Should be in line with reform priorities Must not further entrench corrupt systems Can the performance measured be attributed to specific policy inputs? (‘actionable’)
- 19. Overview of presentation Designing indicators Basic steps Actionable indicators Different indicator approaches Measuring/Monitoring your indicators Potential models/examples
- 20. What indicators measure Corruption Integrity Perception Public opinion Experts Public sector Experience / victimisation General population / vulnerable groups Public sector Private sector Diagnostic Assessments Compliance monitoring Institutions Processes Sectors Local level
- 21. Examples of corruption indicators Number of reported cases of bribery within e.g. the police Ranking of most corrupt institutions in public national perception surveys Bribe payments as % of income among individuals in low-income neighborhoods Proportion of sanctions to registered complaints
- 22. Corruption indicators can measure… The incidence of corrupt transactions experiencial and perception-based data The incidence of sanctions for corruption statistical comparison The impact of corruption data sensitive to vulnerable groups & gender
- 23. Integrity indicators Also called ‘transparency’ or ‘accountability indicators’ Measure the reverse of corruption: what is in place to prevent it Focus on the effectiveness of anti-corruption mechanisms which are within the control of policy makers Therefore tend to be ‘actionable’
- 24. What indicators measure Corruption Transparency/Accountability/Integrity Perception Public opinion Experts Public sector Experience / victimisation General population / vulnerable groups Public sector Private sector Diagnostic Assessments Compliance monitoring Institutions Processes Sectors Local level
- 25. Example of a fact-based / experience-based question In some areas there is a problem of corruption among public officials. During 2011, has any government official, for instance a customs officer, police officer or inspector in your own country, asked you or expected you to pay a bribe for his service? yes no don’t know
- 26. Example of a perception-based question It is known that in some countries the problem of corruption among public officials is perceived to be high by citizens. Imagine a person who needs something that is entitled to him/her by law. Is it likely or not likely that this person would have to offer money, a present or a favour (e.g. more than the official charge) to get help from: Member of Parliament? Elected municipal councillors? Customs officials? Tax/revenues officials?
- 27. Raise your hand every time you agree with a statement: Perceptions are better indicators of progress than facts Facts are better indicators of progress than perceptions Both perceptions and fact-based indicators are useful Neither is helpful, it is not possible to measure corruption
- 28. Drawbacks of perception data Seen as biased, leading some to minimize its use, or rule it out entirely Politically sensitive, especially in polarized political climates, where the media is seen to have undue influence in magnifying public perceptions of corruption Lag effect: perceptions often lag behind reforms
- 29. But perception data matters! High perceptions of corruption are correlated with low perceptions of state legitimacy High perceptions can fuel corrupt practices by: encouraging people to believe they must pay bribes reducing the likelihood of citizens reporting leading those with power to believe that there is nothing wrong with accepting bribes. Perceptions and opinion reveal important information about how corruption works in a specific context
- 30. What indicators measure Corruption Transparency/Accountability/Integrity Perception Public opinion Experts Public sector Experience / victimisation General population / vulnerable groups Public sector Private sector Diagnostic Assessments Compliance monitoring Institutions Processes Sectors Local level
- 31. Inputs and outcomes Inputs (de jure, laws) vs. Outcomes (de facto, practice)
- 32. de jure indicators vs. de facto indicators De jure: In law, is there an agency with a legal mandate to address corruption? De facto: In practice, is the anti-corruption agency effective? Does the ACA make regular public reports (e.g. to legislature)? When necessary, is the ACA able to independently initiate investigations? Can citizens access the anti-corruption agency? Does the ACA act on complaints within a reasonable time period? Can citizens complain to the ACA without fear of recrimination?
- 33. Using de jure and de facto indicators together Combine de jure and de facto indicators to show discrepancies between: change in law & resources ($, infrastructure, procedures, staff) and change in practice (‘improved governance’ in practice)
- 34. Discrepancies in law and practice are revealing Are laws and organisational reforms translating into impact on the ground? If not, why not? Inappropriate reforms? Economically perverse incentives? Vested political interests? Lack of capacity (e.g. human resources, information retrieval systems)? High tolerance of corrupt conduct by the public? Different understandings of what is corrupt conduct?
- 35. Why generate poverty and gender sensitive indicators Corruption exacts a higher price on women and the poor. Rich/middle class men tend to be in positions of power, and thus are beneficiaries of corruption. Need to be sensitive to effectiveness of anti-corruption mechanisms on these groups Indicators too often tend to be gender and poverty blind
- 36. 4 ways to make indicators sensitive to vulnerable groups Disaggregating by poverty/gender Specific to the poor/women Implicitly poverty/gender sensitive Chosen by the poor/women
- 37. Methods of measurement Objective analysis Statistics Surveys Quantifying the qualitative
- 38. Using benchmarks for Action Plans (vs. baselines) It’s not just what you measure, it’s what you measure it against . Benchmarks are imperative to judge progress, demonstrate change, measure success. Use international treaties; standards from the World Bank, TI, etc.; or country-specific objectives
- 39. Developing a scale to quantify qualitative indicators Example of qualitative indicator: In practice, are major public procurements effectively advertised? Quantification: Score 10: There is a formal process of advertising public procurements. This may include a government website, newspaper advertising, or other official announcements. All major procurements are advertised in this way. Sufficient time is allowed for bidders to respond to advertisements. Score 5: There is a formal process of advertising but it is flawed. Some major procurements may not be advertised, or the advertising process may not be effective. The time between advertisements and bidding may be too short to allow full participation. Score 0: There is no formal process of advertising major public procurements or the process is superficial and ineffective.
- 40. How are the tools above applicable to the RBM life-cycle approach?
- 41. Applying the tools to the RBM life-cycle approach? Planning Evaluation Monitoring Actionable Actionable Corruption Corruption Integrity Integrity Experience-based Fact-based Fact-based Experience-based Perceptions Perceptions Impact Impact Output Output Satisfaction Satisfaction Participatory process
- 42. Exercise: Selecting Strategy and Action Plan indicators Goal: To learn how to identify impact and output indicators for long-term and short-term results
- 43. Strategy Goal 1: A decrease in corruption in the judiciary Step 1: Who are the stakeholders who could provide information on this goal? Citizens with contact with the judiciary Lawyers Civil society organizations that work on legal issues Civil servants with contact with the judiciary Step 2: What questions could you ask of citizens to help monitor this goal? Have you had contact with the legal system in the past 12 months? If so, were you asked to pay a bribe? How much have you paid in bribes to judges, lawyers, or other legal officers in the last 12 months?
- 44. Strategy Goal 2: A decrease in corruption in the judiciary ( perceptions) What questions could you ask of citizens to help monitor this goal? Do you think corruption in the judiciary has decreased over the past x years? In your opinion, do you think there is a higher, equal, or lower level of corruption in the judiciary as compared to [insert another key institution]? In your opinion, do you think there is a higher, equal, or lower level of corruption in the judiciary as compared to [insert neighboring country]?
- 45. Monitoring the Action Plan Objective: Increase in transparency of the legislature What indicators would you propose to monitor this objective? What percentage of legislative proceedings are published on a publicly available website? What percentage of public requests for legislative information under the FOI law are granted? What is the average length of time it takes for an FOI request to be granted?
- 46. Benchmarking the Action Plan Indicator for Monitoring No compliance: 0%-30% of legislative proceedings are published Partial compliance: 30%-90% of legislative proceedings are published Full compliance: 90%-100% of legislative proceedings are published What percentage of legislative proceedings are published on a publicly available website?
- 47. Evaluating impact (longer term) Corruption data and Integrity data Experience- based and Perception based Actionable/not Baseline and Benchmarks >> Goal to assess BUT ALSO to feeds into next planning stage
- 48. Don’t reinvent the wheel! Many useful tools are already out there, they just require some adaptation.
- 49. Thank you! Asmara Achcar Regional Specialist, Governance Assessments [email_address]
Editor's Notes
- #6: Simplistic model – not all countries adopt such a linear approach. Timing of assess: Driven to a large extent by assess purpose If goal of assess is to influence public debate about a country’s state of democracy, useful to consult election calendar! If goal is narrower, such as constitutional reform, best to synchronize assess with a schedule already established for public consultation on this reform. If goal is to evaluate reforms or programmes already underway, best to fit assess with timetable officially laid down at outset of reforms in question.
- #7: Anchored in NDS, anti-corruption strategy, APRM, etc. Focus on national gov priorities 3. Conforms to global standards in terms of technical rigour – collect data from variety of sources (surveys, admin data, national stats, civil society) 4. And assess process and results should form part of wider social dialogue (research team comprised of measurement experts from academia and NSO & steering committee – guidance throughout & responsible for presenting results – legitimacy / credibility of assess) 5. Making it publicly accessible 6. Sensitive to vulnerable groups (Framework doc) 7. incl. govt policymakers, civil society, academia, media, parliament, political parties – with particular emphasis on national stats agency in charge of gov data collection, storage & analysis Key capacities required in country-led assess: Facilitating multi-stakeholder discussions on assessment & monitoring Coordinating data-producing agencies Research & data collection: surveys, FGDs, etc. Data disaggregation / analysis Database management of gov data Disseminating / presenting assess results Using gov in policymaking 8. Responsive to current gov priorities 9. Dissimination also to vulnerable groups 10. To enable monitoring of improvement / deterioration in quality of DG
- #8: Based on experience from other countries, these are some of the benefits that may take place as a result of broadening ownership. For example, if stakeholders are involved in making key decisions in creating a framework, such as identifying needs and purpose, select tools, choose relevant indicators, design a method for collecting data, this creates greater commitment. Experience show that ownership and commitment may reduce bureaucratic resistance to the assessment exercise. Technical benefits More likely to be customized to country specificities May facilitate continuous customization of tool May make indicators more “actionable” Political benefits People usually commit to what they help to create More likely to be seen credible’ by policymakers Increases legitimacy and public trust in the exercise May strengthen consensus-building and political will Efficiency benefits (usage) More likely: To be adapted to actual measuring needs That results will be better integrated in planning Indicators are used for accountability It will be sustainable
- #10: "Stakeholder management is critical to the success of every anti-corruption strategy. By engaging the right people and institutions in the right way, you can make a big difference " How are the threatened project targets being used? By whom? Who is threatening the conservation target? • Who is most dependent on the resources at stake? Is this a matter of livelihood or economic advantage? Are these resources replaceable by other resources? • Who possesses claims – including legal jurisdiction and customary use – over the resources at stake? Are several government sectors and ministry departments involved? Are there national and/or international bodies involved because of specific laws or treaties? • Who are the people or groups most knowledgeable about, and capable of dealing with, the resources at stake? Who is managing these resources? With what results? • Are the stakeholders and their interests geographically and seasonally stable, or are there migration patterns? • Are there major events or trends currently affecting the stakeholders (e.g., development initiatives, land reforms, migration, population growth)? • Has there been a similar initiative in the region? If so, to what extent did it succeed? Who was in charge and how did local stakeholders respond? • Who is directly responsible for decisions on issues important to the project? • Who holds positions of responsibility in interested organizations? • Who is influential in the project area (both thematic and geographic areas)? • Who will be affected by the project? • Who will promote/support the project, provided that they are involved? • Who will obstruct/hinder the project if they are not involved? • Who has been involved in the area (thematic or geographic) in the past? • Who has not been involved up to now but should have been?
- #11: This is shared in hard copy in an A3 format, one to each group
- #12: Hypothetical example
- #16: Module 1 - Introduction to the Programme Which, if any, of these indicators are actionable ? (drawn from CPI and WB’s Control of Corruption indicator)
- #17: Module 1 - Introduction to the Programme
- #18: Module 1 - Introduction to the Programme What are the pitfalls of actionable indicators? “ Simply because something can be measured does not mean that it is an important constraint on good governance.” Actionable does not mean action-worthy E.g. Measuring whether a country has an independent anti-corruption commission when there is no guarantee it will reduce corruption E.g. Measuring the speed of judicial proceedings when not clear that increasing the speed will ensure that justice is done Risk of measuring things because they are easily measurable, leading to “teaching to the test” and “reform illusion” Measuring whether a country has an independent anti-corruption commission when there is no guarantee it will reduce corruption (problem might lie in low wages of civil servants – what can independent anti-corruption agency do about that?!)
- #19: Module 1 - Introduction to the Programme Is it measuring interventions that are truly ‘beneficial’? Assessing performance of interventions only makes sense if these are first deemed appropriate for the particular country context Prior research can identify what reforms to prioritize, and ones that risk further entrenching corrupt systems e.g. political economy studies such as National Integrity Systems assessments by TI Can the performance measured be attributed to specific policy inputs? (‘actionable’ indicator)
- #21: Module 1 - Introduction to the Programme This is a map of corruption assessment tools. It classifies these tools generally according to what they measure. On the corruption side, tools differ in their predominant focus (perception or experience), and by respondent group On the anti-corruption side, overall diagnostic assessments, compliance monitoring (e.g. UNCAC, Accession to EU political criteria), Under diagnostic: institutions (e.g. ACA’s, parliaments, judiciary), processes (e.g. procurement, budget), sector specific (e.g. education, health, natural resource management), local level tools (e.g. municipal governance)
- #22: Module 1 - Introduction to the Programme Here are some example corruption indicators - What is the difference between these? Do you see any logical way of grouping them?
- #23: Module 1 - Introduction to the Programme
- #24: Module 1 - Introduction to the Programme focus on integrity instead of corruption because corruption is inherently hidden and therefore hard to measure.
- #25: Module 1 - Introduction to the Programme This is a map of corruption assessment tools. It classifies these tools generally according to what they measure. On the corruption side, tools differ in their predominant focus (perception or experience), and by respondent group On the anti-corruption side, overall diagnostic assessments, compliance monitoring (e.g. UNCAC, Accession to EU political criteria), Under diagnostic: institutions (e.g. ACA’s, parliaments, judiciary), processes (e.g. procurement, budget), sector specific (e.g. education, health, natural resource management), local level tools (e.g. municipal governance)
- #26: Module 1 - Introduction to the Programme
- #27: Module 1 - Introduction to the Programme Note formulation
- #28: Module 1 - Introduction to the Programme We will do a recount after the session!
- #29: Module 1 - Introduction to the Programme e.g. Polarized society – no independent public opinion – no matter what question, respond in favor or against ruling party
- #30: Module 1 - Introduction to the Programme perceptions of corruption affect the public's image of the state. So even if perceptions of corruption are a distortion of the true level of corruption, they are important for shaping popular trust (or distrust) in govt institutions. the public's perception of the extent of corruption also affects the actual level of corruption. Perceptions matter because agents base their actions on their perceptions The more widespread the public believe corruption to be, the more likely they are to believe corruption to be a social norm and to offer bribes to facilitate services from the state. this makes perceptions of corruption important, irrespective of whether they correctly reflect the true extent of corruption. Important info about how corruption works in a specific context: What do you think would happen if you or the person didn’t pay? Would make no difference to the service; would get a bad service; would be threats/harassment; would be delay/denial of service Do you think the following ways of fighting corruption are effective or ineffective? (more prosecutions, increase salaries, more ways for citizens to report corruption, more protection for people who report corruption…)
- #31: Module 1 - Introduction to the Programme This is a map of corruption assessment tools. It classifies these tools generally according to what they measure. On the corruption side, tools differ in their predominant focus (perception or experience), and by respondent group On the anti-corruption side, overall diagnostic assessments, compliance monitoring (e.g. UNCAC, Accession to EU political criteria), Under diagnostic: institutions (e.g. ACA’s, parliaments, judiciary), processes (e.g. procurement, budget), sector specific (e.g. education, health, natural resource management), local level tools (e.g. municipal governance)
- #33: Module 1 - Introduction to the Programme Input Existence and quality of formal rules found in documents, laws, regulations and the constitution (on paper) Answer the question: “What has been done?” Actionable Say nothing about actual progress (implementation) Outcome indicators Assess deliverables to citizens Answer the question: “Are citizens benefiting from specific institutions and policies?” Lack actionability Measure the improved governance in practice Head of ACA protected from removal without relevant justification & through a formal process? Difficulties accessing politically sensitive info necessary for carrying out investigation? Subject to favourable or unfavourable criticism by govt? Political appointments vs. based on professional criteria? Conflicting family relationships / personal loyalties? Threats / harassments?
- #34: Module 1 - Introduction to the Programme Input Existence and quality of formal rules found in documents, laws, regulations and the constitution (on paper) Answer the question: “What has been done?” Actionable Say nothing about actual progress (implementation) Outcome indicators Assess deliverables to citizens Answer the question: “Are citizens benefiting from specific institutions and policies?” Lack actionability Measure the improved governance in practice Head of ACA protected from removal without relevant justification & through a formal process? Difficulties accessing politically sensitive info necessary for carrying out investigation? Subject to favourable or unfavourable criticism by govt? Political appointments vs. based on professional criteria? Conflicting family relationships / personal loyalties? Threats / harassments?
- #35: Module 1 - Introduction to the Programme Impact on the ground – reduced fees for services, fewer bribery demand encounters, easier access The implications of monitoring both the input and the outcome sides of reform means broadening our data collection methods. (not only review of laws & policies & procedures in place, but actual impact on the ground – through surveys, FGDs, etc.) In the next session we will consider the related issue of objective and subjective data.
- #36: Module 1 - Introduction to the Programme
- #37: Module 1 - Introduction to the Programme Disaggregating by income/sex: The proportion of poor households using public services who experienced corruption directly in the last 12 months in comparison to non-poor households. (Measuring experience of corruption) Specific to the poor/women: Size of funds allocated to legal aid in provincial budgets (per capita) (Measuring inputs) Implicitly poverty/gender sensitive: The proportion of public agencies for which public expenditure tracking surveys (PETS) are regularly conducted. Chosen by the poor/women: The amount you need to pay to get medical assistance (Indicators derived from focus groups)
- #40: Module 1 - Introduction to the Programme How do you assess simple yes/no questions? some indicators require a scale : a set of numerical values assigned to certain criteria for the purpose of quantifying qualitative indicators. Scaling introduces quality benchmarks Example 1: How responsive was Ministry X in disclosing the requested information?
- #43: Groups will identify one of each for the area that they have chosen, and report back to the plenary