SDVC Project Evaluation Design_Establishing Two Counterfactuals
1 like1,353 views
The baseline study for the Dairy Value Chain Project (SDVCP) in Bangladesh, conducted by IFPRI, aims to evaluate the project's objectives, including improving rural milk collection, access to markets, and the overall dairy transport system. It employs a rigorous impact evaluation methodology comparing outcomes for program participants and a control group, using propensity score matching to identify similar households for this analysis. The study involves both quantitative and qualitative data collection from 1,510 households across nine districts to provide insights for potential project adjustments or expansions.
SDVC Project Evaluation Design_Establishing Two Counterfactuals
- 1. Evaluating the Dairy Value Chain Project in Bangladesh: Baseline Study Akhter Ahmed International Food Policy Research Institute Seminar at CARE-Bangladesh Dhaka May 28, 2009
- 2. Storyline SDVCP objectives Evaluation methodology Baseline data Characteristics of survey households Gender related issues Dairy farming practices Qualitative review of value chain actors INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
- 3. SDVCP objectives 1. Improve milk collection systems in rural and remote areas 2. Improve access to inputs, markets, and services by mobilizing groups of poor farmers, producers, and char dwellers 3. Improve the milk transport network 4. Ensure access to quality service at the producer level 5. Improve the policy environment INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
- 4. The baseline study CARE has commissioned IFPRI to conduct a thorough evaluation of the SDVCP that would allow CARE to make informed decision of whether to close, revise, extend, or expand the SDVCP CARE has also contracted the Data Analysis and Technical Assistance Limited (DATA) to collect quantitative and qualitative information for the evaluation, under guidance and supervision of IFPRI This baseline study is a part of the evaluation. It was specifically designed to permit a scientific and rigorous evaluation of impacts of the SDVCP through follow-up studies INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
- 5. Impact Evaluation Methodology Impact is the difference between outcomes (e.g., income, school enrollment, women’s empowerment, etc) with the program and without it The goal of impact evaluation is to measure the this difference in a way that can attribute the difference to the program, and only the program Use Difference-in-differences method that compares observed changes in the outcomes for program participants (treatment) and non-participating comparison group (control), before and after the program Combine with propensity score matching to adjust for pre- program differences INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
- 6. Propensity Score Matching (PSM) Match program participants (treatment) with non- participants (control) at baseline (before intervention) Each program participant will be paired with a non-participant that is similar Use PSM to pick an ideal comparison group from the baseline survey data The comparison group will be matched to the treatment group using “propensity score” Propensity score is predicted probability of participation given observed pre-program characteristics of participants and non- participants INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
- 7. Illustrating Difference-in-Difference Estimate of Average Program Effect PA Impact = (PA - CA) - (PB - CB) Program CA Control PB=CB Baseline Follow-up (Before) (After) INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
- 8. Constructing the counterfactual: Two control groups The control or comparison groups are comprised of eligible but non-participant households Two control or comparison groups of households have been created to assess the impact and to capture the potential spillover effects The nature of SDVCP interventions may generate spillover effect of the project. For example, if new dairy production technologies are introduced, non-beneficiaries may copy these Control 1 households have been selected from unions where the SDVCP is operating Control 2 households have been selected from upazilas without any milk chilling pants in the nine project districts INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
- 9. Baseline data Used both quantitative and qualitative data for the baseline study Quantitative data came from a comprehensive household survey designed by IFPRI and carried out by DATA The survey questionnaire was designed to collect information on multiple topics, including household demographic composition, level of education, school participation, occupation and employment, dwelling characteristics, assets, food and nonfood expenditures, morbidity, economic shocks, anthropometric measurements of children and women, and a detail module on dairy farming practices INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
- 10. Baseline data … The household survey was carried out in 9 SDVCP districts, 27 upazilas, and 60 villages Sample size: Determined by power calculation with design effect. Total sample of 1,510 households, of which 659 program participants, 425 Control 1, and 426 Control 2 households Survey started on August 20, 2008, and completed on September 14, 2008. Data entry completed by end October. Data cleaning, including logical consistency checking and data validation completed by mid-January 2009 Qualitative data collection: Used key-informant interview, focus group discussion, and observation methods INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE