Masterclass Performance Measurement Framework
0 likes748 views
The document outlines a performance measurement framework aimed at assessing the livelihoods, environmental stewardship, and productivity of farmers through a shared set of indicators and metrics. It emphasizes the importance of collaboration among stakeholders to improve data efficiency and effectiveness while addressing common learning questions. The framework includes strategies for monitoring and evaluating agricultural interventions, ensuring that farmers' needs and experiences are adequately captured and evaluated.
Masterclass Performance Measurement Framework
- 2. Shared framework of indicators Impact Areas Guiding Question Indicator Livelihood and Well Being Are the basic needs of the farmers being met? Food Security: Access to sufficient food Income Assets Perceived Well-Being Gender What are women’s roles and benefits in this crop? Women’s participation in crop Equitable Access to Training Participation in Decision-Making Environmental Performance Is the land well stewarded? Adoption of conservation practices Farm Productivity Are farmers realizing the potential of their farm? Adoption of good ag practices Estimated Productivity Crop Revenue or Net Income Access to Services Do farmers have access to services? Access to credit, training and inputs Are farmers using these services? Use of credit, training, and inputs Trading Relationships Are farmers experiencing good trading relationships? TBD
- 3. What is a shared approach? • A common framework of indicators and metrics to help guide the practitioner. • A suite of indicators to pick what’s appropriate. Not a single set of indicators. • Shared approach implies committing to use the same indicators and metrics when asking the same questions.
- 4. Why a shared approach? From shared questions about farmers to shared approach for: • Greater efficiency and effectiveness • Reduced burden on suppliers and farmers • More effective community learning
- 5. Criteria leading to framework • Fewest questions (indicators) that give “sufficient” insight into livelihoods and performance. • Affordability and scalability vs scientific robustness • Simplicity vs nuance • Embedding approaches for regular monitoring and reporting
- 6. Characteristics of Performance Measurement Indicators and metrics appropriate for: • Surveys under 30 minutes • Minimally trained enumerators or even self reporting • Across a wide range of supply chain types Yields actionable data Cost efficient enough to scale
- 7. Grew from overlap in a theory of change many share
- 8. Out of the ToC grew common learning questions
- 9. the Learning Questions dictated the Shared Framework of indicators Impact Areas Guiding Question Indicator Livelihood and Well Being Are the basic needs of the farmers being met? Food Security: Access to sufficient food Income Assets Perceived Well-Being Gender What are women’s roles and benefits in this crop? Women’s participation in crop Equitable Access to Training Participation in Decision-Making Environmental Performance Is the land well stewarded? Adoption of conservation practices Farm Productivity Are farmers realizing the potential of their farm? Adoption of good ag practices Estimated Productivity Crop Revenue or Net Income Access to Services Do farmers have access to services? Access to credit, training and inputs Are farmers using these services? Use of credit, training, and inputs Trading Relationships Are farmers experiencing good trading relationships? TBD
- 11. Scope of the Shared Approach Framework Core reporting metrics (a subset for reporting & communication) Shared PM Framework (detailed enough to be “actionable”) Full Measurement Study (will usually address additional questions)
- 12. Made-to-fit Each organization will have it’s own purpose and goals for measurement, but when we share learning questions in common, we can align on indicators and increase the value of our data by gathering it in a common way.
- 13. M&E in the Programme Cycle Financing and contracting Final Evaluation Implementation Scoping Project Design M&E Strategy/Framework Operational Planning M&E Plan/Matrix Partici pation Monitoring and Evaluation
- 14. Impact Assessment Year 1 Year 2 Year 3 Year 4 Year 5 M&E systems Deep dive Deep dive Year 0 Baseline Impact Assessment Interventions Goal: Evaluate impact of specific changes (interventions) so that outcomes can be attributed to the specific interventions. Valid control groups (counterfactuals) are needed.
- 15. Performance Monitoring Year 1 Year 2 Year 3 Year 4 Year 5 Performance Monitoring Chain wide indexes Goals: Assess “status” and track change over time. Outcome change cannot be attributed to specific causes but trends can be tracked and correlations observed.
- 16. Mixing methods Year 1 Year 2 Year 3 Year 4 Year 5 (2) Performance measurement to track broad annual change on few key indicators (short survey by technician, 10 min) Baseline (1) Baseline to key issues, get stakeholder buy in, and tailor survey to specific context. Research + focus groups Interventions (3) Focused impact Assessment on small sample to answer specific causation question. Trained interviewers plus control group
- 17. Example of thinking process Learning questions: Are the farmers consistently food secure in our supply chain? Is food security improving? Indicators: Food security Specific Metric: % of farmers with 2 months or more of food insecurity Approach: Youth in coop run short interviews during annual meeting using Ipod-based system. Survey question: Are there times in the year where you have to reduce meal size, skip meals, or change diet? Which times in the year? (number of months) What you want to know What you ask a farmer
- 18. Appropriate methodology depends on purpose (and budget!) Purpose: Why do you want to know? What will you do with the results? Who is the audience? Learning questions & population: What questions are you trying to answer? About whom? Indicators, Metrics, and Questions: What are the appropriate indicators to track? What specific survey questions are appropriate and effective? Approach: How and to whom are you going to ask the questions? Embedding: How could this be part of an ongoing system? Purpose Learning Questions ToC Approach Indicators/ survey Embedding
- 19. Of course -- depends on purpose, specific questions, and resources Key Methodology Questions How do we collect the data? From whom do we collect data? Who collects the data? Self reporting Group settingIndirect Household Group gathering point Rough sampling Voluntary individual Statistical sampling w/control Coop TA YouthSelf Certifier Researcher
- 20. Of course -- depends on purpose, specific questions, and resources Key methodology Questions How do we collect the data? Who collects the data? Self reporting Group settingIndirect Household Group gathering point Rough sampling Voluntary individual Statistical sampling w/control Coop TA YouthSelf Certifier Researcher household survey for rough baseline From whom do we collect data?
- 21. Of course- depends on purpose, specific questions, and resources Key Methodology Questions How do we collect the data? Who collects the data? Self reporting Group settingIndirect Household Group gathering point Rough sampling Voluntary individual Statistical sampling w/control Coop TA YouthSelf Certifier Researcher From whom do we collect data?
- 22. Of course- depends on purpose, specific questions, and resources Key Methodology Questions How do we collect the data? Who collects the data? Self reporting Group settingIndirect Household Group gathering point Rough sampling Voluntary individual Statistical sampling w/control Coop TA YouthSelf Certifier Researcher From whom do we collect data?