SIAC program report

Photo: S. Kilungu (CCAFS)
SIAC Program Report
16th
August 2016

of 42
SIAC Program Report: Summary of Progress To-Date
Prepared by SPIA for the SIAC External Evaluation
SIAC, Strengthening Impact Assessment in the CGIAR, is a four-year program of work led by SPIA that aims to
broaden the coverage and widen the range of impact measures of CGIAR research. The program started in
2013 and was motivated by a strong and growing demand across the development community for high-quality
ex post impact assessment to support evidence base decision making in the CGIAR. Expanding the evidence
base, including data collection across the full range of CGIAR research types, was deemed essential in
underpinning and sustaining investments in the CGIAR in the post-reform period. Expanding the inventory of
impact assessment of CGIAR research is one component of that effort; the other relates to enhancing the
credibility and rigor of these studies through greater independence, transparency and external review. Ex post
impact assessment is important in the reformed CGIAR in terms of assessing the strength of the linkages
between agricultural research and the System-Level Objectives (SLOs) of poverty reduction, food security,
improvements in nutrition and health, and sustainable natural resource management. SIAC activities aimed at
developing new methodological tools and collecting the needed data for impact assessment will allow the
CGIAR to extend its work across a broader range of research activities and impact pathways.
The key objectives of the SIAC program are as follows:
 Objective 1 (Methods): Develop, pilot and verify innovate methods for collection and assembly of
diffusion data;
 Objective 2 (Outcomes): Institutionalize the collection of diffusion data needed to conduct critical
CGIAR impact evaluations;
 Objective 3 (Impacts): Assess the full range of impacts from CGIAR research;
 Objective 4 (Building a community of practice): Support the development of communities of practice
for ex post impact assessment within the CGIAR and between the CGIAR and the development
community more broadly.
A major feature of the work being carried out in this program, and most particularly for the activities under
Objective 3, is the sponsorship through a competitive grant-making process of studies of economic, social, and
environmental impacts. All else being equal, employing external and independent researchers ensures a higher
level of objectivity and credibility, compared with relying on CGIAR centers to carry out these studies.
Nevertheless, CGIAR centers and researchers often have key knowledge and, in the majority of cases, are
involved to a varying extent (depending on capacity) in many of the studies.
The program is now in its fourth year but is likely to extend another six months to allow time to bring to
completion a number of studies (see Annex 1 for list of funded studies and timetable). The four-year budget
for this program of work is approximately US$ 12 million1
. This report summarizes the progress to-date and is
organized around the Objectives spelled out in the SIAC program of work.
1 Funding for SIAC for the period 2013 – 2016 comes from three donors:
- USD 5,238,799 from the Bill and Melinda Gates Foundation (BMGF);
- USD 4,453,057 from CGIAR Window 1 and
- USD 1,923,568 from the CGIAR ISPC.

of 42
Underpinning this objective is the development of a robust set of methods for routinely tracking adoption of
CGIAR-related technologies in a cost-effective manner. Such information is a prerequisite for achieving the
highest quality assessment of outcomes and impacts. A set of activities are designed to test innovative ways of
assessing the adoption of improved varieties of crops, livestock and fish technologies, and agronomic and
natural resource management interventions, with the goal of eventually embedding protocols derived on these
tests into large-scale surveys carried out by other institutions outside the CGIAR, such as the World Bank’s
Living Standards Measurement Survey – Integrated Surveys of Agriculture (LSMS-ISA). There are four main
areas of activity here which are managed by Michigan State University.
Activity 1.1. Advance methodologies for tracking the uptake and adoption of improved varieties
The objective of this Activity is to pilot test and validate alternate approaches to collect variety-specific
adoption data against the gold standard benchmark (DNA fingerprinting) to determine which
method/approach is the most cost-effective (i.e., which method provides a given level of accuracy at the least
cost). The idea is to come up with ‘lessons learned’ and recommendations on methods / approaches that can
be used in scaling up the collection and assembly of diffusion data on improved varieties. The following crop-
by-country combinations were targeted:
1. Cassava in Ghana;
2. Maize in Uganda
3. Beans in Zambia
Two further crop-by-country combinations were added to the SIAC portfolio
by SPIA at the start of 2015, as part of the collaboration with LSMS-ISA:
4. Cassava in Malawi
5. Sweet potato in Ethiopia
1. Cassava in Ghana:
This study tests the effectiveness of the following four household-based methods of tracking varietal adoption
for cassava against the benchmark of DNA analysis of cassava leaf samples.
A. Elicitation from farmers by asking him/her the: 1) names of varieties planted and some basic
questions for each variety planted; and 2) type of variety (improved vs. local)
B. Farmer elicitation on varietal characteristics by showing a series of photographs (or actual plants).
This information will be later used by the analyst to identify varieties based on morphological
characteristic data.
C. A trained enumerator recording observations on varietal characteristics by visiting the field and
sharing their opinion on what the variety is by: 1) name and 2) by type (based on observations). The
information collected will be also used by the analyst to identify varieties based on morphological
characteristic data.
D. Enumerator taking photos of the plant in the field for latter identification by experts (i.e., breeders)
The field work for this study is jointly supported by SIAC and the RTB CRP and conducted in partnership with
IITA, Crops Research Institute (CRI)-Ghana, and Agriculture Innovation Consulting (AIC) Ghana. Field work was
completed in late fall 2013. All the samples collected from the farmers’ fields and the 40 genotypes included
in the reference library were sent to IITA by the Ghanaian partners in January 2014. DNA extraction work for
almost 1000 samples was completed by IITA and all the samples were shipped to Cornell for Genotyping by
Sequencing (GBS). Data from the GBS analysis were submitted by Cornell to IITA in July 2014. But due to some
issues on limited library materials to classify all the farmer samples, more analysis was required that included
cassava genotypes from IITA’s collection. The data from this updated analysis were made available to IITA in
December 2014.
A presentation summarizing the main results of this case study along with the results of the bean study (case
OBJECTIVE 1: Develop, pilot and verify innovative methods for collection and assembly of
diffusion data (METHODS)

of 42
study 3, below) were presented at ICRISAT in June 2015 (upon their invitation during a visit by M. Maredia), at
the AAEA meetings in July 2015, and at the International Conference of Agricultural Economists (ICAE) in August
2015. A research paper summarizing the DNA fingerprinting methodology and results of this study was
published in BMC Genetics (v. 16:115 DOI 10.1186/s12863-015-0273-1).
The proportion of farmer collected samples classified as released/improved varieties using different methods
ranged from 1% for method A1, to 15% for method D. The implication of this is that there could be a large
variation in the estimates of adoption of improved varieties based on which method is used to derive the
estimate. Additionally, several interesting findings about the materials included in the reference library have
emerged from this case study. The most interesting results were that: 1) some released varieties included in
the reference library were genetically identical (e.g., IFAD and UCC); 2) many released varieties were hybrids
or admixtures; and 3) Library accessions representing both ‘released varieties’ and ‘landraces’ fall under the
same varietal cluster groups. This last finding especially posed a challenge for varietal identification. The
problem it created is: How to classify farmer samples that fall in these three variety cluster groups? Should
they be classified as ‘improved/released’ varieties or local/landrace varieties?
To address this dilemma, the analysis of effectiveness of methods (A to D) against the benchmark of DNA
fingerprinting is done under two scenarios / assumptions: Liberal scenario: which assumes that all the farmer
samples that fall in a variety cluster in which there is at least one released variety are essentially improved
varieties. Under this assumption, 31% of farmer samples are classified as improved materials; and Conservative
scenario which assumes the opposite (i.e., farmers samples that match the DNA results of a variety group in
which there are both released varieties and landraces, the variety group is considered not-improved). Under
this scenario only 4% of farmer samples are classified as improved materials.
In both the liberal and conservative scenario, when the results of methods A - D are compared with DNA
analyses, the results indicate that: 1) a large number of farmers are mistakenly identifying varieties as improved
varieties when it was not or identifying a variety as traditional when it was in fact improved; and b) the methods
of varietal identification that relied on ‘experts’ were better than the farmers’ elicitation, but still way off from
the truth established by the DNA fingerprinting method. At least at the variety level, the results of this study
has clearly demonstrated the unreliability of both farmer and expert elicitation based methods of varietal
identification.
2. Maize in Uganda:
As part of the planned DTMA (Drought Tolerant Maize in Africa) adoption survey by CIMMYT in three districts
in Eastern Province of Uganda, MSU had designed and implemented modules and protocols to test the
effectiveness of the household-based methods of tracking varietal adoption for maize similar to those used for
the cassava in Ghana experiment. Field data were collected in June 2014 and leaf tissues from 416 maize fields
across 34 villages were collected for DNA analysis. The National Crops Research and Resource Institute
(NaCRRI) of NARO served as the ‘technical’ partner for DNA analysis through their ongoing project with the
University of Ghana. Due to delays in transferring the leaf tissues from the field to the lab and to the large
amount of compacted leaf material in the tubes, virtually all the samples were lost due to mold development.
Due to the delays and difficulties experienced during this project, LGC Genomics offered to repeat the work
for this project for free of charge before June 2015 (for 34 sample plates x 146 assays). An alternative was
found with SPIA to piggy back on a planned LSMS experiment on maize in Uganda in 2015, and management
of the study was transferred over from MSU to SPIA in March 2015 (see below). Regarding the survey data
received from CIMMYT, the survey results indicate that, in general, farmers do not have a clear or accurate
idea about what types of maize varieties they are growing and very few (6%) were able to show the bags in
which the seed planted was obtained (most share or purchase seed with neighboring farmers).
Since March 2015, under SPIA, the context for the study has now shifted to a large methods experiment run
by the World Bank LSMS-ISA team and UBOS on estimating maize productivity – the Methodological
Experiment on Measuring Maize Productivity, Varieties, and Soil Fertility (MAPS). The following three methods
for varietal identification were embedded in the design of the experiment:

of 42
A. Asking the farmer to identify the variety.
B. Asking the farmer to answer questions related to 15 phenotypic characteristics (using a visual aid),
checked against sets of reference responses for each variety using alternative decision rules.
C. Focus group meeting with a number of experts.
These will be benchmarked against two DNA genotyping methods:
D. DNA fingerprinting using SNP markers on samples from maize leaf tissue (using the credit with LGC
Genomics under their contract with MSU/NaCCRI)
E. DNA fingerprinting using DArT method of genotypingon samples from maize grain.
Expert opinion elicitation for these two districts (Iganga and Mayuge) was also carried out prior to any field
work taking place.
Field work for the whole survey took place over three visits to a sample of 900 households (post-planting; crop-
cutting; post-harvest) over the period April 2015 – August 2015 in 5 districts in Uganda. For budget reasons,
DNA fingerprinting was possible only on a subset of 550 farms in two districts – Iganga and Mayuge.
Enumerators from UBOS were recruited and trained intensively for one month, and survey data collection was
facilitated by the use of networked tablets for real-time data management and processing. Leaf samples were
collected at the post-planting visit in April and May 2015, from within the quadrant laid down by enumerators
for subsequent crop-cutting, using leaf collection kits from LGC Genomics. Grain samples were subsequently
collected from these quadrants in the follow-up crop-cut visit in June and July 2015. SNP-based genotyping
data was received from LGC in September 2015 following analysis of the leaf samples. Grain samples were
processed (dried, ground to flour, labelled) by NACCRI in August and September 2015, and shipped to Diversity
Arrays in Australia at the end of October 2015.
Both genotyping methods (D and E) have been successfully applied and the results are currently being
compared by colleagues at Diversity Arrays and NaCCRI. Early results show that the SNP-based genotyping used
on the leaf samples was insufficiently discriminating among improved varieties – some of the varieties in the
reference library appear genetically identical when screen against a low number of SNPs (approx. 140),
whereas when a highly quantitative assay used in DArt is applied (examining more than 10,000 alleles), then
genetic distance is observed.
Before any consistency with the correct genotype (correct identification) can be established, we have to
consider whether unique identification is possible for a given method. This is possible only for 47% of farmer
responses, and 13% of responses to questions in the morphological protocol. Farmers submit “don’t know”
responses to the open question of what the variety is that they are growing in 53% of cases. In theory, a set of
11 morphological questions is sufficient to uniquely discriminate among all varieties in the reference library.
However, in practice, farmers are clearly unable to respond to the morphological questions with sufficient
accuracy to allow for unique identification. Only 60 (13%) of the response sets from farmers interviewed
correspond completely to a valid set of responses (as determined by characterization of the reference library).
How many of these 60 response sets (from method B) are consistent with the genotyped identification (as
opposed to occurring by chance from the set of multiple choice questions) is currently being investigated, as is
the consistency with the expert opinion elicitation process.
Only 2% of the sample of 477 farmers are able to correctly identify the variety when checked against the
genotype (method E). Method E results show that every farmer sampled (all 477) is growing an improved
variety of maize – an astonishing finding. Furthermore, there is deep penetration of the commercial varieties
from Western Seed company (WE varieties) and YARA seeds, whereas farmer’s expectations are largely
confined to the LONGE series released by the government NARO.
While this suggests that improved materials are reaching farmers’ fields some way or another, there is also a
deeper, less clear picture that emerges when one considers the heterogeneity of the samples. There is
considerable genetic heterogeneity within the samples used for the reference library. A good cut-off for
acceptable heterogeneity in the reference material is 15%, whereas our mean reference library heterogeneity
level is 33%, suggesting that genetic lines have not been well separated in the breeding process, or the seed
sampled for the reference library was not pure breeders’ seed. Second, there is a very low average purity level
of the field samples. Only a minority of field samples are at 80% purity or above, dropping sharply and bringing

of 42
the average purity to 63%. This means that for an average plot, 37% of the genetic material sown is not from
the primary improved variety planted in the plot, but has been introduced to the plot either deliberately
(through counterfeiting, or from a farmer choosing to combine varieties together in the same plot) or through
handling / labelling errors in the seed supply chain.
The MAPS experiment also has first-rate data on agricultural productivity, soil quality, varietal identification,
and household characteristics. We can estimate some simple models for the determinants of productivity:
once using the data that are typically collected in surveys, based on farmer testimony; and once using objective
methods for varietal identification (DNA fingerprinting), yields (crop-cuts), soil quality (soil samples taken and
analysed in laboratory). These results will help us understand more about the importance of data quality in
context – does improved data quality substantially impact on our understanding of some fundamental issues
in impact assessment? A second round of MAPS is currently in the field between June and September 2016.
SPIA is considering have the second round of crop cuts genotyped, as a follow-on from the first round.
Four academic articles are in various stages of preparation with SPIA involvement (John Ilukor and/or James
Stevenson), drawing on various elements of the MAPS experiments.
3. Beans in Zambia:
This study tests the effectiveness of the following four household-based methods of tracking varietal adoption
for common beans.
A. Elicitation from farmers by asking him/her some basic questions for each variety planted.
B. Showing the farmer seed samples representing different varieties and asking him/her to identify the
sample that matches each of the variety grown on their farms.
C. Collecting seed samples representing each variety planted by farmers for latter identification by
experts (i.e., breeders).
D. Enumerator taking photos of the seeds during the survey for latter identification by experts (i.e.,
breeders).
The accuracy of adoption estimates derived from the above four methods was evaluated against the varietal
identification established through DNA fingerprinting of seed samples collected from the farmers.
The context for this study was a bean adoption study conducted by PABRA (Pan-African Bean Research
Alliance), in collaboration with CIAT and the Zambian Agricultural Research Institute (ZARI). Seed samples and
data corresponding to four methods, similar but not identical to the cassava in Ghana study, have been
collected from 402 households that were surveyed under a PABRA study, thus allowing to leverage survey
costs.
During a visit to Zambia, the MSU team visited the local market (in Kasama) and collected some bean seed
samples from local vendors. These seeds were added to the pool of seeds for DNA analysis to check if the seeds
of varieties sold in the market as named by the ‘vendors’ match the actual variety as named and identified by
farmers. The total seed sample (both collected from the farmers and from the vendors) was about 900.
As a next step the seeds collected from the farmers’ fields and from the market were germinated by the ZARI
breeder in June 2014. A technician from CIAT-Uganda traveled in July to Zambia to help with the DNA extraction
and samples were shipped to LGC Genomics. To establish the library, ZARI included all the released varieties
plus 15 other local materials in the samples shipped to LGC.
Overall, the adoption (or more appropriately, the frequency of use by farmers) of improved variety as a group
based on farmer elicitation methods varied substantially from 15% in method A (asking farmers what type of
variety they planted—local or improved) and 72% in method B (showing farmers seed samples). The estimate
of overall adoption of improved varieties based on the two expert elicitation methods--method C (showing
breeders seed samples) and method D (showing breeders’ seed photos) were similar. The estimated adoption
(or frequency of use) was 36% based on method D and 37% based on method C.
Comparing the two methods based on farmer elicitation, results indicate that there was only 25% agreement
on the name of the variety planted between methods A and B. In the case of the two methods based on
experts’ opinion (i.e., methods C and D), there was close to 80% agreement on identifying the varieties either

of 42
by name or by type. The raw data from the SNP analysis were received from LGC Genomics Lab in May 2015.
These data were shared with CIAT researchers for interpretation and a report of the data analysis was
submitted by CIAT soon after. Based on these data and report from CIAT, MSU has completed the data analysis
to test the effectiveness of different methods of varietal identification.
Preliminary results show DNA fingerprinting benchmark of 16% adoption, with farmer elicitation method A
returning estimates of 4% adoption of improved varieties when asked for by name, and 13% when asked for
as the aggregate class (local vs improved). Method B of showing farmer seed samples resulted in an estimate
of 71% improved variety adoption, whereas showing seeds or photos to breeders resulted in adoption
estimates rather close in aggregate (though with many possible mis-matches at level of individual varieties, at
18% for showing photos and 15% for showing seeds.
A report summarizing the results of the two pilot studies in Ghana (with cassava) and Zambia (with beans) is
under preparation and will be soon published as a MSU Staff Paper. In addition, a manuscript targeted to a
peer reviewed journal is being developed and will be ready for submission by the end of August.
4. Cassava in Malawi
The context for this study is a methods experiment led by Talip Kilic of the World Bank LSMS-ISA based around
alternative approaches to estimating cassava production from households using diaries and different lengths
of recall data. SPIA, through Research Associate John Ilukor, have embedded the following varietal
identification approaches into the design of the experiment:
A. Asking the farmer to identify the variety.
B. Asking the farmer to answer questions related to phenotypic characteristics (using a visual aid),
checked against sets of reference responses for each variety using alternative decision rules.
C. Focus group meeting with a number of experts.
These are benchmarked against:
D. DNA fingerprinting using GBS on samples from cassava leaf.
Cassava leaf collection was integrated in the survey, along with a phenotypic protocol of traits, which was
implemented starting June 2015 and continued through to end of June 2016. Leaf collection for fingerprinting,
and phenotypic data for subjective identification, are complete. The reference library of varieties, and their
corresponding phenotypic attributes, was compiled by the Malawian NARS. DNA was extracted in-country by
the Chitedze laboratory for samples from 1002 farms and shipped to Diversity Arrays in Australia for
sequencing.
In terms of unique identification, virtually all (1001 or 99.9%) of the farmers submitted a response to the
question as to which variety was being grown (method A), whereas unique identification was only possible for
23% of the sample using method B, and indeed only 73% of the sample using genotyping (method D). The low
level of unique and valid responses for method B is likely a combination of error on the part of farmers but also
the fact that the number of local varieties is very long and as many as 130 local varieties were not included in
the reference library. This large number of distinct local varieties without samples held by the NARS for
inclusion in the reference library also explains the relatively low (73%) rate at which the genotyping exercise
was able to find a match between samples and a valid reference.
In terms of correct identification, farmer elicitation (method A) gives an adoption rate for improved varieties
of 19%, method B suggests a 70% adoption rate, whereas the genotyping (method D) actually shows only 10%
adoption of improved varieties.
5. Sweet potato varietal identification in Ethiopia
This experiment was the initiative of SPIA Research Associate Frederic Kosmowski, working in Ethiopia with
locally recruited enumerators and contacts through the NARS system in Ethiopia. The objective was to assess
the accuracy of three household-based methods for identifying sweet potato varieties using DNA fingerprinting
as the benchmark. The methods used were:
A. Elicitation from farmers with basic questions for the most widely planted variety;

of 42
B. Farmer elicitation on five sweet potato phenotypic attributes by showing a visual-aid protocol
C. Enumerator recording observations on five sweet potato phenotypic attributes using a visual-
aid protocol and visiting the field
D. DNA fingerprinting using GBS on samples from sweet potato leaves (as the benchmark).
Data were collected in early 2015 from 259 plots in Ethiopia. Leaf samples were taken, DNA was extracted by
ILRI in Addis, and plates for sequencing shipped to Diversity Arrays in November 2015. The reference library
was collected and has been complemented by sequencing accessions from the CIP genebank. Initial analysis
shows the following results:
 Method D (DNA analysis) finds that 63% of the farmers are cultivating improved varieties and 37%
are using Local varieties;
 Method A produces almost identical results in the aggregate: across all responses 64% of the cultivars
are self-identified as improved, 35% as local and 1% unknown.
 However, 30% of the actual (DNA-based) improved varieties were identified by farmers as Local and,
of the 85 actual Local varieties in the sample, slightly more than half were wrongly identified as
improved.
 Variety names (for both improved and Local) given by farmers delivered inconsistent and uncertain
varietal identities and only 4% of the farmers were able to correctly identify the registered names of
specific improved varieties.
 Visual-aid protocols employed in methods B and C were more accurate than method A, but still far
below the adoption estimates given by the DNA fingerprinting method.
Results here suggest that estimating the adoption of improved varieties of sweet potato in this area of Ethiopia
with methods based on farmer self-reports is not reliable and indicate a need for wider use of DNA
fingerprinting, which is likely to become the gold standard for crop varietal identification across many crops
and countries, subject to further testing.

of 42
Table 1 - Comparison of methods for estimating % adoption of improved varieties in SIAC program (DATA COLLECTED – SUMMARY BY SEPT 2016)
Focus (Study team – SIAC
activity)
Sample size Corresponding
expert opinion
estimate
Farmer:
Asking
for
variety
name
Farmer:
Asking
“Improved
or local”?
Farmer:
Showing
range of
reference
seeds
Farmer:
Series of
questions
on
phenotypic
attributes
Experts: Visit
field and
identify by
specific
variety name
Experts:
Visit field
and identify
“Improved
or local”
Experts:
Enumerator
takes photo
and experts
identify
Experts: Seeds
collected from
farms and
shown to
experts
DNA
Reference
Cassava, Ghana (MSU / IITA –
Activity 1.1)
914
(all methods)
36%
(DIIVA, 2009)
1% 6% 2% 5% 15% 4% – 31%
Beans, Zambia (MSU / CIAT –
Activity 1.1)
Between 736
and 855
(varies)
9.5%
(DIIVA, 2009)
4% 13% 71% 18% 15% 16%
Maize, Uganda – Leaf (SPIA /
Diversity Arrays – Activity 1.1)
550 How to
handle
don’t
knows?
46% Didn’t
uniquely
identify..
Pending
from
Andrzej
Maize, Uganda – Grain (SPIA
/ LGC – Activity 1.1)
550 Ditto 46% Ditto 100%
Sweet potato, Ethiopia (SPIA
/ Diversity Arrays – Activity
1.1)
259 4% 64% Ditto 63%
Cassava, Malawi (SPIA /
Diversity Arrays – Activity 1.1)
1,200 61%
(DIIVA, 2009)
How to
handle
don’t
knows?
19% 70% 10%
Wheat, Bihar
(MSU / CIMMYT / ICRISAT –
Activity 2.1)
3,278
Lentil, Bihar
(MSU / CIMMYT / ICRISAT –
Activity 2.1)
1,003
Cassava, Vietnam
(MSU / CIAT – Activity 2.1)
1,000
Rice, Indonesia
(MSU / IRRI – Activity 2.1)
810
Cassava, Nigeria (IITA –
Activity 3.1)
GIFT Tilapia, Philippines
(World Fish – Activity 3.1)

of 42
Activity 1.2. Develop protocols for tracking diffusion of natural resource management technologies
A call for pilot projects under this activity was issued by MSU in July 2013 and two studies were commissioned:
1. Innovative use of mobile phone based applications in tracking adoption of Natural Resource
Management Technologies in India (CIMMYT), and
2. Hyperspectral signature analysis: a proof of concept for tracking adoption of crop management practices
in Gazipur, Bangladesh (IRRI)
The final technical reports for the two competitively selected pilot studies funded under this Activity were
received by MSU in April 2015 (from CIMMYT) and June 2015 (from IRRI). The reports were reviewed by SPIA
and MSU, and comments of this review were shared with the authors. SPIA is publishing an impact brief on the
CIMMYT study to highlight the results and lessons learned on the application of an Integrated Voice Response
System to track the adoption of resource management technologies and farming practices.
Overall, the pilot study on the mobile phone based Interactive Voice Response System (IVRS) method was
inconclusive about the validity of results collected through the IVRS model compared to paper based survey
results. But this study has helped identify several key lessons that can guide future applications of such
methods. Some of these insights/lessons gained from this pilot are:
 Monitoring the adoption trends is a dynamic and time consuming process. The use of IVRS can help
reduce some of the time and money costs.
 The IVRS technology used to collect data on adoption resolves the issue of scalability as it can be
inclusive of all types of locations with mobile penetration, and does not have bias towards the type
of phone handsets and service providers thus increasing the reachability especially in rural
environments.
 The application of the IVRS based survey is neutral to any of the agricultural or natural resource
management technologies and practices. The set of survey questions and the question sequence can
be easily adapted on any technology and its adoption and can be customized in the system.
 Farmers do appreciate the limited time that they have to spend with IVRS as compared to the long
paper surveys done conventionally. However, the use of this method requires basic literacy and
understanding of the use of mobile phones.
 The technology is simple, customizable, and scalable. But it is suitable only for short surveys (i.e.,
focused on only one technology at a time).
 The project had envisioned the risk on validity of the data collected through IVRS if the farmers are
not able to understand the purpose of this survey and disconnect the phone. Thus it is important to
run pilots and awareness creation on how to respond to the IVRS based survey.
For the IRRI study, the research team was unable to obtain hyperspectral imagery for the test site (as initially
proposed) and resorted to using alternates (Landsat 8 and MODIS). SPIA has a number of concerns regarding
the study, in particular whether this is the right type of remote sensing imagery that should be used. The
lessons from these two pilots have significantly informed our strategy for Activity 2.2 implementation, which
was the intention when the SIAC program was designed. Indeed, a contract to Nong Lam University and UC
Santa Cruz has been awarded under that activity to apply alternative remote sensing approaches to assessing
AWD adoption in Vietnam. Regarding cell-phone surveys, we have understood more about the biases from
phone surveys and constraints regarding assembling a sample frame, and regarding remote sensing, through
having this work externally reviewed, we have understood more about the heterogeneity of remote sensing
approaches and the strengths and limitations of different methods.

of 42
3. Measuring crop residue cover in Ethiopia
This pilot study that was added in 2015, but is directly managed under SPIA. The experiment was the initiative
of SPIA Research Associate Frederic Kosmowski, working in Ethiopia with locally recruited enumerators and
contacts through the NARS system in Ethiopia. The study is testing six alternative methods of crop residue
coverage measurement among a sample of rural households in Ethiopia. These methods are compared against
a benchmark, the line-transect method. The alternative methods compared against the line-transect include:
A. Interviewee (respondent) estimation;
B. Enumerator estimation visiting the field;
C. Interviewee with visual-aid without visiting the field;
D. Enumerator with visual-aid visiting the field;
E. Field picture collected with a drone and analyzed with image-processing methods
F. Satellite picture of the field analyzed with remote sensing methods.
The survey experiment was implemented in five enumeration areas located in the sub-humid areas of East and
West Shewa zones. In each enumeration area, 12 panel households from the Ethiopian Socio-Economic Survey
(ESS) were interviewed. In addition, 28 households were randomly selected to participate in the experiment.
Data collection took place in late 2015 and early 2016. Low-cost drones (Phantom 2+) were used to capture
aerial pictures of the surveyed fields. Close supervision ensured collection of high quality data and resulted in
a total sample of 197 households and 314 fields observed. After survey completion, two archived full scenes
of Landsat 8 Thematic Mapper satellite imagery were acquired from the United States Geological Survey’s
Earth Explorer imagery search and delivery website. These images match the dates (late 2015 and early 2016)
associated with each field location. A Normalized Difference Tillage Index was calculated for each field. The
experiment delivered the following results:
 Survey-based methods (A, B, C, D) tend to underestimate field residue cover.
 In continuous analysis, suing boxplot and scatterplots, correlations with the line-transect benchmark
ranged from -0.25 (for Method E) to 0.76 (for Method D). The best, i.e., closest to the benchmark,
estimates came from the two visual-aid protocols.
 In categorical analysis (>30% cover or not), visual-aid protocols (C and D) and the remote sensing
method (F) perform equally well with greater than 80% accuracy.
 Among survey-based methods, the strongest correlates of measurement errors are total farm size,
field size, distance and slope.
 Results deliver a ranking of measurement options and suggest a wider use of visual-aid protocols
among survey practitioners and researchers for accuracy, cost and ease of implementation. Remote
sensing provides a good measure only in terms of categorical analysis but is much harder to
implement for statistical institutes.
Activity 1.3. New institutional approaches to collecting technology diffusion data
Most diffusion surveys in the past have depended on CGIAR research teams, either working on their own or
working in collaboration with national programs and statistical services to generate the data. In many
countries, there are private market research firms as well as private survey firms engaged in carrying out
household surveys for academic purposes. A call for proposals was issued by MSU with a focus on doing a case
study in India. The call was issued in February 2015, and applied to either for-profit or non-profit entities with
the relevant capacity.
A total of six proposals were received and after review carried by MSU, proposals received from two private
sector firms based in New Delhi (Synergy Technofin and Creative Agri-Solutions Private Limited-CASPL) and one
firm based in Chennai (Nathan Economic Consulting India Private Limited) were recommended to SPIA for
funding. After receiving an approval from SPIA, MSU established Letters of Agreement with the three firms to
undertake the pilot studies to test the innovative approaches. The scope of these pilots is outlined below:
1. Led by Synergy; Haryana (Karnal) and Bihar (Vaishali); Technologies: Zero till, direct seeded rice, LLL;
Wheat-rice based farming systems
2. Led by CASPL; Haryana (Karnal) and Punjab (Ludhiana); Technologies: Zero till, direct seeded rice, LLL;

of 42
Wheat-rice based farming systems
3. Led by Nathan; AP (Anantapur and Kurnool); Technologies: 15 soil conservation measures promoted by
ICRISAT; Groundnut farming system
To validate the estimates of technology adoption to be obtained from the three pilot studies, MSU has
conducted representative surveys in the 5 study districts by using a more ‘traditional’ approach. This approach
consists of working with a survey firm in India (identified from a process of issuing expressions of interest) to
help with the logistics of doing data collection. The questionnaire and sampling design was developed by MSU
with little involvement of the contracted survey firm. But the survey firm provided enumerators (hired
specifically for this survey), organized training for the enumerators (with one MSU PhD student actively
participating in the training of enumerators and making sure all the field activities are planned as per the survey
design), took charge of programming the survey questionnaire as a CAPI survey, provided logistical support to
the field staff, and did data quality checks, data verification, and submitted the data set to MSU.
The main idea behind this pilot was to see if agricultural technology adoption data collection can be outsourced
to survey firms at lower cost and with no loss in quality. In particular, the approach tested the use of locally
based enumerators equipped with tablets. It was envisioned that this would reduce cost by minimizing
enumerator travel costs.
The results on cost are mixed. The 4,000 household validation survey, conducted by MSU and used as a
comparison for this approach, cost 22 USD per household. The cost for the three survey firms ranged from12
USD to 43 USD. There are several reasons, however, to believe that the local enumerator approach may still
ultimately be more cost effective. One is that each of the survey firms that participated in the pilot study
developed their own Android based data collection application with this budget. While the application may
require tweaking on a survey by survey basis, that amount will be significantly lower than the development
cost, reducing pre household data collection costs. The second reason is that the scale of the validation survey
(five districts rather than two per survey firm) allowed for data collection and management cost efficiencies
that could have lowered the average cost. Additionally, validation survey deployed less than half the number
of enumerators per household, reducing enumerator training costs per household. Finally, the local
enumerator approach is likely to be more cost effective for reoccurring data collection such as for annual
technology adoption. After the first round of data collection, the cost per household via the local enumerator
approach will likely drop due to reduced travel cost and from the survey site.
The primary issue with the implementation of the local enumerator approach was that android application
development, testing and refinement took longer than anticipated. And because this was the first survey using
the newly developed application for each of the implementers, the applications were not as polished as they
may be for subsequent data collection. This means that built in validations and logic checks are not as accurate
as they may be for subsequent rounds of data collection. This had a negative effect on data quality as data still
required extensive cleaning. This includes for errors such as impossibly high values that are easily avoidable
with a more refined app. A more refined application will also reduce the impact of enumerator ability on data
quality. This is especially important as local, rural enumerators may have less experience and education than
more urban-based conventional enumerators. However in this study at least, results on key enumerator
characteristics are mixed. The comparison of enumerator characteristics across the approaches piloted suggest
that at least in India it is possible to find people in rural areas with as good or better qualifications as
conventional enumerators. India, however may be relatively unique in this respect. In less developed countries
it will be more difficult to find qualified, rural based enumerators. This then makes it even more important to
use well developed data collection applications that reduce the effect of enumerator characteristics on data
quality.
All the data collection for this pilot study have been completed and received by MSU. Data analysis to compare
the estimates of adoption of focused technologies using the two approaches (local enumerator vs. validation
survey) is currently ongoing..
Activity 1.4. Develop and disseminate best practices for collecting diffusion data
The idea with this activity is to take stock of activities, results and lessons learned from activities 1.1, 1.2 and

of 42
1.3, in order to generate guidance for the CGIAR system more broadly. This activity will be organized in the
form of a workshop and earlier discussions with SPIA and PIM (F. Place) has resulted in the decision to organize
this workshop in Boston post-AAEA meetings (August 3-4, 2016). This workshop will bring together
results/learning from SIAC Objectives 1 and 2, and other private and public sector partners involved in finding
innovative ways to collect technology adoption in developing countries. The main objectives of this workshop
are to:
1. Take a stock of current and innovative methods for measuring adoption of agricultural technologies
2. Share and discuss results and insights from pilot studies and experiments conducted to establish proof
of concepts to harness the potential of new methods for tracking adoption of agricultural practices
and other types of technologies
3. Further the discussion on scaling up proven methods for measuring technology adoption
We are expecting more than 40 participants to attend this workshop. Information about the workshop,
including the draft agenda and the participant list, is available from the SPIA website. A document summarizing
the main outcomes of the discussion of this workshop will be developed as a deliverable of this Activity.

of 42
The objective here is to compile and make available the best information on outcomes that are at least
plausibly attributable to CGIAR research outputs, and on a large-scale. This is where a key bench-marking
function for the CRPs is most obviously fulfilled by this program. Large gaps in existing adoption databases for
genetic improvement technologies (activity 2.1), natural resource management technologies (activity 2.2) and
policy-oriented research (activity 2.3) will be filled for priority regions. In addition, under activity 2.4, the World
Bank Living Standards Measurement Study-Integrated Surveys of Agriculture (LSMS-ISA) team and SPIA and
Centers are working together with NARS partners and statistical agencies to see how some of these processes
can best be integrated into existing surveys to reduce cost and increase frequency of data collection. MSU is
exploring similar objectives in Zambia and Mozambique and in dialogue with Indian counterparts for a similar
objective.
Activity 2.1. Organize the collection of crop germplasm improvement research related direct
outcomes
Under the SIAC project objective 2, this Activity (2.1) has expanded on the DIIVA and TRIVSA projects that have
come to a closure, and focus on the collection of varietal diffusion data in South and Southeast Asia.
MSU is leading a process for which varietal release and varietal adoption data are collected for 62 crop x
country combinations (CCCs) (which increases to 130 if we count individual states within India and regions
within China as equivalent to countries – they all have their own data collection efforts) using expert opinion
elicitation methods. Towards the planning of Activity 2.1, a two day inception meeting with Center and NARS
partners was held in Bangkok on January 15-16, 2014 for a total of 35 participants. Based on the discussion
and input from resource persons and participants, a guideline document on the methodology for collecting
varietal release and varietal adoption data using expert elicitation methodology was finalized by MSU and
shared with all the Centers and NARS partners. Subsequent to the inception meeting, each participating Center
prepared a budget and workplan, upon which MSU established sub-contracts with the centers to collect
varietal release and adoption data (using expert elicitation method) for 130 CCCs (see Table 2).
For 3 CCCs (all legume crops), MSU will work directly with NARS to collect the information. These include
chickpea in Pakistan, and Lentil in Bangladesh and Nepal. For the former two CCCs, MSU has identified and
contracted local NARS partners (NARC in Pakistan and BARI in Bangladesh) to collect the information and
develop the two datasets by mid-2015. The NARS partners in Nepal were also contacted for their assistance in
completing this Activity for lentil crop. But they have not been able to give their commitment to complete this
task. MSU is working with the ICARDA researcher based in India to find an appropriate partner to collect this
data for Nepal.
For the work contracted to CGIAR Centers, activities have progressed as per the plan. Towards the
implementation of this Activity, CIMMYT organized a training workshop in August 2015 in Nepal for the NARS
coordinators. Sushil Pandey and M. Maredia participated as trainer and resource person at this workshop. A
similar training workshop was planned by IRRI in September 2014 in Laos, by ICRISAT in October 2014 in India,
and by CIP in China in February 2015. Sushil participated in all these training workshops as a trainer and
resource person. M. Maredia and T. Kelley participated in the India workshop in October 2014. CIAT has
identified a regional economist to lead this Activity working closely with the NARS coordinators in each CCC.
Since the last report, Centers have made significant progress in completing the data collection for their
targeted numbers of CCCs. Table 3 provides a summary progress report on work accomplished and still pending
towards completing data collection for the two databases (varietal release and varietal adoption). Overall, data
collection to compile the two databases has been completed for all but 3 CCCs. This represents an overall
achievement of 98% of the targeted numbers of CCCs. Three centers have completed the data collection for
100% of their CCCs, and have also submitted the technical reports and the two databases. The LOA for ICRISAT,
CIP and IRRI has been extended till end of July or August and they plan to submit the datasets for the remaining
CCCs over the next month. According to the last progress report received from CIP, they will have completed
all EE workshops by August 9.
OBJECTIVE 2: Institutionalize the collection of the diffusion data (OUTCOMES)

of 42
Table 2: Final list of CCCs as per the workplan submitted by Centers (July 2014) – amended in September 2014
Country Rice Maize Wheat Barley Sorghum
Ground-
nut
Chick-
pea
Pigeon
pea
Lentil Cassava Potato
Sweet
potato
ALL
Afghanistan 1 1
Bangladesh 1 1 1 1 1 5
Bhutan 0
Cambodia 1 1 1 3
China (provinces
listed below)
8 8 6 2 1 12 9 46
India (states listed
below)
4 8 6 4 4 2 6 3 37
Indonesia 1 1 1 1 1 1 6
Iran ** 0 0 0 0 0 0
Laos 1 1 2
Malaysia 1 1
Myanmar 1 1 1 1 1 5
Mongolia 0
Nepal 1 1 1 1 1 5
Pakistan 1 1 1 1 1 5
Papua New Guinea 1 1
Philippines 1 1 1 1 4
Thailand 1 1 1 3
Vietnam 1 1 1 1 1 1 6
Total 21 25 17 5 0 5 3 1 7 10 23 18 130
Lead center IRRI CIMMYT ICRISAT CIAT CIP/RTB
Commitment from
lead center
21 39 15 10 41 127
Gap * 3 0 3
*MSU will work directly with national programs or consultants to get information for the two data base for these 3 CCCs.
** Due to US Government’s restrictions on ‘working’ with Iran, five CCCs have been removed from MSU’s workplan and LOA.

of 42
Table 3. Data collection status by Centers, as of the end of July 2016.
Total
mandated
CCCs
Data collection in mandated CCCs
Center Completed
To be
completed
Percentage
completed
Report
submitted
Databases
submitted
CIMMYT 40 40 0 100% Yes 40
CIAT 10 10 0 100% Yes 10
IRRIa 21 21 1 100% Yes 21
ICRISAT b 15 15 0 100% No 11
CIP 41 39 2 95% No 18
MSUc 3 2 0 67% Yes 2
Total 130 127 3 98% 102
a For Indonesia, the NARS collaborator plans to implement the EE method to collect adoption data for Upland rice. This will be in
addition to the adoption estimates obtained for Lowland rice in Indonesia using the seed sales data.
b ICRISAT has indicated they will share the outputs of 2 additional CCCs (groundnuts in China) for which they are collecting the data
using similar methodology. Similarly, CIP may submit data from Bhutan for potato, in addition to the 41 CCCs in their workplan
c Adoption data for one CCC -- Lentils in Nepal will not be completed due to non-response from the local NARS partner identified by
ICARDA last year.
MSU, in consultation with SPIA had identified the four CCCs for doing validations of adoption estimates to be
derived using expert elicitation method or secondary data sources. Two methods are being used for
validation—estimating adoption using representative farmer surveys and DNA fingerprinting on all or a sub-
set of seed samples. The four CCCs identified for validation of Activity 2.1 are:
 Wheat in Bihar (state level)
 Lentil in Bihar (state level)
 Cassava in Vietnam (country level)
 Rice in Lampung Province, Indonesia (province level)
Field work (data and sample collection) for all these CCCs has been completed. Samples have been finally
received by the labs (ICRISAT lab for wheat and lentils; IRRI lab for rice; and CIAT lab for cassava) and genotyping
analysis is currently undergoing. No DNA fingerprinting results have yet been received for these validation
CCCs.
As part of the audit requirement, and to collect opinion and assessment of experts on the elicitation
methodology used in Activity 2.1, MSU has designed a survey (see Annex 2), which is being sent to all the EE
workshop participants. This email and online survey is being facilitated by each Center focal point with the help
of their national coordinating partners. The results of this survey will help assess the approach used to collect
crop varietal adoption data across different CCCs, and to get some additional information on the status of the
seed system for specific crops. Hopefully, the feedback from this survey will help us improve this methodology
of collecting adoption data in future studies.
Activity 2.2. Organize the collection of natural resource management (NRM) research outcomes
This was initially part of the Michigan State University sub-grant but it was agreed in Jan 2014 that SPIA would
manage this part of the program. Following a delayed start after this work was transferred back to SPIA, a call
for Expressions of Interest was finally issued in October 2015, for case-studies focused on the following
priorities NRM practice – country combinations:

of 42
Table 4 – Priority NRM practice-country combinations for call for EoIs issued October 2015
PRIORITY NRM PRACTICES PRIORITY COUNTRIES
AGROFORESTRY (PARTICULARLY
“FERTILIZER TREES”, LEGUMINOUS
FODDER SHRUBS)
Kenya, Zambia, Zimbabwe, Rwanda
ALTERNATE WETTING AND DRYING (AWD)
IN RICE PRODUCTION SYSTEMS
China, Vietnam, Philippines, Indonesia, Myanmar, Bangladesh
CONSERVATION AGRICULTURE IN MAIZE-
BASED SYSTEMS
Zambia, Zimbabwe, Mozambique, India, Pakistan, Nepal, Bangladesh,
Kyrgyzstan, Uzbekistan, Tajikistan, Turkmenistan, Kazakhstan, Iraq, Mexico
COCOA INTEGRATED CROP AND PEST
MANAGEMENT (ICPM)
Cameroon, Cote d’Ivoire, Ghana, Liberia, Nigeria
MICRO-DOSING OF FERTILIZER IN MAIZE-
BASED SYSTEMS
Kenya, Zimbabwe, Mozambique
INTEGRATED SOIL FERTILITY
MANAGEMENT
Kenya, Rwanda, Burundi, DRC
From this call, 62 expressions of interest were received, and these were scored and review by SPIA in November
2015. Proponents from 18 expressions of interest as well as a number of resource people and SPIA secretariat
members were invited to participate in a workshop in Rome in December 2015 comprising: discussions of the
nature of the priority practices; the existing data infrastructure in place in the relevant countries that can serve
as the basis for generating adoption estimates; prospects for remote sensing; and group work clustered around
the six practices. The overall objective of the workshop was to try and broker collaborations across interested
parties to ensure we got a strong set of full proposals.
Following the workshop, SPIA issued an invitation to the workshop participants specifying a set of 9 work
packages that full proposals should be targeted towards. Proponents were invited to outline “core” and
“upgraded” budget options for their proposals, with sets of activities to match. In February 2016, the 12 full
proposals received (together covering a total of 25 of our practice-country combinations) were externally
reviewed by a five-member expert panel, and a recommendation for funding proposals was put to the PSC for
discussion and decision on 17th March 2016. Work by the proposal teams takes place throughout the
remainder of 2016 and run to mid-2017. Hence, this is one of the activities that has made the no-cost extension
to mid-2017 necessary. The following contracts are now in place for work being carried out between April
2016 and June 2017.
Table 5 – Funded NRM practice-country combinations with institutions and methodological approaches
TEAM NRM PRACTICE COUNTRIES METHOD(S)
MUTENJE ET AL, CIMMYT /
ICRISAT
Conservation agriculture Mozambique,
Zambia
Panel methods
HOLDEN ET AL, NMBU Conservation agriculture Malawi Panel methods, lead farmers
and followers
ARSLAN ET AL, FAO (EPIC
TEAM)
Conservation agriculture and
Agroforestry
Zambia, Malawi Coordination across all other
CA projects, analysis of
secondary data
MAVZIMAVI ET AL, ICRISAT
/ UIUC
Conservation agriculture and
micro-dosing
Niger,
Zimbabwe
New surveys

of 42
BUTLER ET AL, IFMR / U
MICH
Conservation agriculture India - Bihar,
UP, Haryana,
Punjab
New sur vey + remote sensing
SONDER ET AL, CIMMYT Conservation agriculture Mexico Remote sensing
LOVELL ET AL, NONG LAM
UNIVERSITY / UC SANTA
CRUZ
Alternate Wetting and Drying Vietnam Remote sensing
VAGEN ET AL, ICRAF Fertilizer trees and fodder shrubs Zambia Remote sensing + HH survey
NKONYA ET AL, IFPRI /
GEOPOLL
Integrated soil fertility
management
Zambia, Kenya,
Rwanda
Panel (Zamiba, Kenya) + SMS
survey (Rwanda)
Paul Vlek has been appointed as a senior consultant to help guide this set of studies over the period of
implementation, working with the SPIA team - James Stevenson and Nuri Niyazi in particular. A results
workshop will be organized in May / June 2017.
Related to the documentation of NRM outcomes, James Stevenson represented SPIA in a workshop held in
Cairns, Australia in June 2015, on assessing the effectiveness of landscape level interventions. The consensus
in the group was that there is too little attention paid to demonstrating whether, and under what
circumstances, a landscape scale approach is beneficial and will bring about impact. A paper reflecting these
ideas, led by Jeff Sayer, is under review at the journal Ecology and Society.
Activity 2.3. Organize the collection of policy-oriented research outcomes
This activity focuses on another under-assessed area of CGIAR research – policy oriented research, in particular,
identifying intermediate outcomes of CGIAR research that bear on macro level policies and practices plausibly
linked to Center outputs. Work under this Activity attempts to document several categories of policy research
related to:
 Agricultural and relevant macro, trade and nutrition/health policies, all of which can have a large
impact on economic incentives in agriculture, as well as modulating the poverty and nutrition impacts
of some new technologies Management practices/protocols/agreements adopted at national or
international levels
 Levels and types of investments in agricultural research, roads, markets and other infrastructure
 Expansion of training and institutional capacity (e.g., through farmer field schools)
 Major international conferences / workshops around a highly relevant theme, e.g., IFPRI’s 2020 Vision
conferences
Activity 2.3, therefore, focuses on outcomes of CGIAR policy-oriented research (POR) that have influenced
significant policy changes related to agriculture, food and nutrition at the regional, national or global level. The
aim is to compile and make available to CGIAR stakeholders the best available information on outcomes that
are, at least plausibly, attributable to CGIAR policy research outputs. Ultimately, the objective is to build an
inventory of CGIAR policy-oriented research outcome claims that have been externally vetted and passed
minimum plausibility test, as a basis for selecting more in depth case studies of influence and impact could
(through Objective 3 type activity).
In the first phase completed in 2014, consultant Mitch Renkow drew on earlier CGIAR PMS data files from 2006
through 2010 to compile a list of 93 outcome statements that credibly describe significant achievements of
‘deriving from Center POR outputs’. For each POR outcome, information is provided on the constraint or
problem that was addressed, the key research outputs underpinning the outcome, a description of the specific
POR outcome itself, what supporting evidence exists, and the region or country in which the outcome took
place. Sixty-one of these were assessed as Category I “strong” cases – ones that satisfied certain specific
criteria. In addition to the 61 strong outcomes, there were 32 other outcome statements that were deemed
to have significant potential but required further documentation to be considered plausible cases of influence.

of 42
Of the latter 32 outcomes, 17 were judged to require additional evidence linking the outcome to specific Center
outputs, e.g., the existing outcome statement provided insufficient information to make a compelling case that
the policy outcome could be reasonably attributed to the Center. Fifteen additional statements described
outcomes that look promising, but either were at an early stage, e.g., described early outcomes emanating
from pilot projects, or were simply not described well enough to make a strong case for being a POR outcome
– but, again, appear to have good potential to generate meaningful policy outcomes. The categorization draws
on: original Science Council commissioned external reviewers’ evaluations and the consultant’s own
judgement about the strength of evidence/logic.
Phase 2, also led by Renkow, focused on updating the 2006-2010 database, primarily by searching the websites,
annual reports and other relevant documents published by Centers and CRPs between 2011 and 2014, and
applying a similar set of criteria to potential cases of POR influence/impact. This resulted in an updated (2011-
2014) inventory of plausible case study outcomes. Typically, though with a few exceptions like IFPRI, there
were much fewer cases to report over this latter period and with much less information to substantiate the
stories, presumably due to the lack of any strong incentive to produce evidence of outcomes – compared to
earlier years. A follow-up activity, but not as yet undertaken in SIAC Phase 1, entails offering Centers the
opportunity to verify earlier submitted information or provide updated information to substantiate or modify
earlier claims in the phase 1 & 2 inventories. That activity will take place in the Fall 2016, with the possibility
of initiating an external validation process of POR outcome claims assembled under Phases 1 and 2. The latter
may also feature in a SIAC Phase 2.
Two other Activity 2.3 related outputs are worth noting here:
 IFPRI, the PIM CRP and SPIA recently co-sponsored a Workshop on Best Practice Methods for Assessing
the Impact of Policy Oriented Research at IFPRI HQ in Washington DC. The workshop brought together
more than 40 people, including evaluation experts from within CGIAR, the academic community,
donors, and developing country policymakers. The workshop format was designed to foster the
expression of different perspectives on the current state and prospects of impact assessment of POR.
One of the workshop’s objectives was to seek agreement on realistic expectations for what can and
cannot be achieved in evaluating the impact of different types of policy research, and how best to
undertake the work. Key findings of the workshop can be found on the IFPRI website:
https://www.ifpri.org/publication/workshop-best-practice-methods-assessing-impact-policy-
oriented-research-summary-and
 Renkow authored and presented a paper on ‘assessing the impact of policy-oriented research in the
CGIAR: methodological challenges and reasonable expectations’ at the International Conference on
Impacts of Agricultural Research – Towards an Approach of Societal Values (French National Institute
for Agricultural Research INRA, Paris, November 3-4, 2015). The paper offers a critical assessment of
efforts by the CGIAR and kindred national agricultural research institutions to evaluate the welfare
impacts of policy-oriented research conducted under their auspices.
Activity 2.4. Long-term institutionalization of collection of adoption data
SPIA’s long-term vision in achieving this objective is to involve a broader and more diverse set of national
institutional partners in the collection of adoption data so as to systematize the collection of nationally
representative data (on a regular basis) in the most cost-effective way possible. MSU is working in India,
Mozambique and Zambia to explore the integration of technology adoption data into existing surveys. On a
parallel track, SPIA is working with the World Bank Living Standards Measurement Study – Integrated Surveys
of Agriculture (LSMS-ISA) team through two researchers – Frederic Kosmowski and John Ilukor.
1. India (MSU):
The initial efforts (meetings and discussions) focused on ICAR to leverage existing data or future data collection
efforts (cost of cultivation data) for the purpose of tracking and monitoring the adoption of improved varietal
technologies (and any other technologies, if data are available) by farmers on a regular basis. While there was
some initial interest, subsequent interactions suggested that ICAR did not have institutionalized data collection
mechanism in place to integrate this data, and a better target for such efforts might be the Ministry of

of 42
Agriculture or National Sample Survey Organization (NSSO) or to try and work at the state level (in 1-2 states)
and see if the Department of Agriculture in a given state is open to this idea of institutionalizing the collection
of technology adoption data at least on a pilot stage.
Since the SIAC update in February 2015, Mywish Maredia traveled to Odisha, India, in May 2015 for a day, and
visited the Department of Economics and Statistics for the State of Odisha to find out more about the types of
agricultural data being collected at a state level. From this visit and the desk review of questionnaires used to
collect different types of data through surveys that are routinely conducted (such as the crop cut experimental
data, input surveys, agriculture census surveys and NSSO surveys), the emerging conclusion is that India is a
data rich country. There is an impressive amount of data being routinely collected (many at representative
scale), and all these efforts are already institutionalized within the government system. However, despite these
efforts, the fact remains that it is not easy to get an overall representative picture and trend of the adoption
of different types of agricultural technologies that are generated by the Indian research system (and the
collaborating CGIAR centers) due to a number of reasons, including government confidentiality laws.
Due to the characteristics of the way data are collected, processed and reported in India, there is limited utility
of these data for tracking technology adoption at a representative scale. There is certainly room for
improvements in this data system, but a local institution or a research center needs to champion this cause.
The goal would be to make some changes in the institutionalized data collection system so that the data
collected using public resources can serve the research and monitoring needs of the agricultural research
communities. MSU has initiated a conversation along these lines with the National Institute of Agricultural
Economics and Policy Research (NIAP/ICAR), and will continue to pursue these efforts: NIAP/ICAP Director has
written to the Secretary of Agriculture to make household unit level data available to researchers, and intends
to approach the Chairman of the Statistical Commission. However, to date, we have not been able to make
any meaningful progress towards our objective with this work. The reason being that the Director of NIAP with
whom MSU had initiated the discussions has left NIAP and according to the new Director the chances of
influencing any change in the current data collection efforts is highly unlikely in the short time frame of the
SIAC project.
2. Mozambique (MSU):
MSU has liaised with the Directorate of Economics and Statistics (DEST) within the Ministry of Agriculture and
Food Security (MINAG) that is responsible for producing official agricultural statistics. The Integrated
Agricultural Survey (IAI) is a routine data collection effort – representative at the provincial level – and done
every 1-3 years. Last year, MSU reviewed the IAI survey instruments and provided feedback on integrating
some technology specific questions in different sections of the survey. However, DEST was unable to
incorporate all the suggestions as it was planning to conduct only a “light” round of IAI last year. They have
also expressed interest in testing new methods of tracking adoption of varietal technology, especially using
DNA fingerprinting, but no concrete plans emerged on implementing this method due to resource constraints.
No other activities or plans for institutionalizing data collection were discussed or planned in Mozambique.
3. Zambia (MSU):
MSU reviewed the Crop Forecast Surveys (CFS) that is conducted annually by the Ministry of Agriculture &
Livestock and Central Statistical Office. This survey is representative of small and medium scale holdings at the
country level. Suggestions for modifications and addition of a one page section on the adoption of conservation
technology were made to the CFS coordinator – this was pilot tested in February 2015, but was not
implemented in the March-April round of CFS due to time constraints (increased survey length and time).
However, the team has agreed to integrate a page of questions in the second follow-up round (post-harvest
season in September-October 2015). During a recent visit to Zambia (on another project), MSU (M. Maredia)
visited the Ministry of Agriculture and Livestock to get an update on this activity. All data collection has been
completed and currently undergoing data entry and cleaning. Once the data are cleared by the Central
Statistical Office, it will be shared with MSU and broader research community to assess the adoption of
conservation technologies at the national level. This is an example of a successful outcome from this process
of engagement with country statistics agencies.

of 42
4. Ethiopia (SPIA and World Bank LSMS-ISA):
The third wave (2015/16) of the Ethiopia Socioeconomic Survey (ESS) presents an opportunity for integrating
a number of questions related to the adoption of CGIAR-related agricultural technologies. The ESS is a
nationally representative survey of 4,000 households, and is managed by Central Statistics Agency (CSA) via a
network of some 300 resident enumerators.
SPIA were able to incorporate additional adoption-related questions into the ESS for the following
technologies: Orange-fleshed sweet potato; Awassa variety sweet potato; Crop rotation in previous three
years; Treadle pump; Motorised pump; Desi / Kabuli type of chickpea; Weather index insurance; Broad-bed
maker; Improved livestock feed module. Data collection is complete, but we can expect to have access to the
data in September 2016 – ahead of the formal release in 2017.
5. Uganda (SPIA and World Bank LSMS-ISA):
The Annual Agricultural Survey (AAS) is a new survey funded by the Ugandan government and implemented
by the Ugandan Bureau of Statistics (UBoS). The survey instruments were pre-tested in the second season of
2015 and the main survey will start in September / October 2016. SPIA were able to incorporate questions into
the AAS for the following technologies: bean varieties; cassava varieties; maize varieties; sweet potato
varieties; sorghum varieties; agroforestry; livestock; conservation agriculture. John Ilukor has played an
important role in testing the questionnaires, and SIAC funds paid for 20 tablet computers for UBoS to use in
the survey.
In Uganda, the fourth wave of the Integrated Household Survey (the true LSMS-ISA panel survey) has been
delayed, but a second round of the maize experiment (MAPS, described in Activity 1,1) has is currently in the
field.
6. Malawi (SPIA and World Bank LSMS-ISA):
In Malawi, the Integrated Household Survey 4 (LSMS-ISA panel survey) is taking place in 2016. Training began
in February 2016, and fieldwork started in late March 2016. John Ilukor and James Stevenson, with input from
the FAO EPIC team, have introduced questions on a number of NRM practices into the survey instrument,
relating to inter-cropping, crop residue management, agroforestry, crop rotation. John Ilukor helped in training
enumerators with the Malawian National Statistics Office.

of 42
While work under Objectives 1 and 2 paves the way for future ex post impact assessment studies, Objective 3
activities are focused on carrying out a number of impact assessments of CGIAR research and development
initiatives along the entire chain of causation - from research investments to the System-Level Outcomes. Since
this causal chain is long and complex, SPIA is approaching it from a number of different perspectives: case
studies that focus on measuring the impact of CGIAR research on health and nutrition (activity 3.0); long-term
large-scale studies of impact for major areas of CGIAR investment (activity 3.1); sets of micro-scale impact
studies using experimental methods (activity 3.2) to provide evidence on the impact of CGIAR research-derived
technologies to adopting households; studies of a number of under-evaluated areas of research, e.g. irrigation
and water management; livestock, agroforestry and biodiversity (activity 3.3); a system-level meta-analysis of
ex post IA of CGIAR research (activity 3.4).
Activity 3.0. Assessing the impacts of agricultural research on nutrition and health
Evidence of causal linkages between agricultural research and effects on health and nutrition is anecdotal at
best, and yet the demand for such evidence has never been stronger. This activity is motivated by the need to
broaden and deepen the evidence base regarding the potential for agriculture research and development to
leverage health and nutrition benefits, and to improve our understanding of the multiple pathways linking
those two variables. The intention is to complement, not to duplicate, on-going work in the A4NH and other
CGIAR Research Programs, and giving priority to areas that until now are relatively “under-evaluated.” This
prominently includes activities related to measuring the impact of research-derived interventions that
plausibly impact on nutrition and health.
A competitive call for case studies was issued in July 2013. Led by Erwin Bulte at Wageningen University, an
external review team identified an interesting portfolio of studies with different methods and focal
technologies. An inception workshop for the five funded studies was held in July 2014 and since late 2014, we
have had the five studies running as follows:
1. Adoption of high iron bean varieties in Rwanda (CIAT, Harvest Plus, Virginia Tech, Rwanda Agric Board)
The study is assessing the adoption and nutrition impacts of High Iron Bean (HIB) delivery efforts. This involves
verifying the adoption of HIB varieties in Rwanda and then comparing bean consumption and iron intakes of
adopters to those of non-adopters of HIB varieties. Two cross-sectional surveys of bean growers in Rwanda are
planned in order to collect adoption and in depth socioeconomic and nutrition data from a sample of randomly
selected 91 communities and 1104 bean growing farmers in Rwanda. The impact of HIB delivery interventions
on nutritional outcomes, i.e. bean consumption and iron intake, will be assessed using two methods:
propensity score matching and an instrumental variable approach. Cost-effectiveness of HIB delivery
interventions will be calculated by comparing costs of delivery of HIB to the health benefits of the intervention
(measured in terms of DALYs saved).
Progress: A progress report received from the team in December 2015 demonstrated that the project has
overcome some logistical difficulties and is progressing well. Erwin Bulte has been providing ongoing support
to the team to try and ensure they identify a good instrumental variable for their analysis, and that the follow-
up survey rounds in 2016 include dietary diversity and food security modules. Household and community
surveys were completed in 2015, and DNA fingerprinting will take place during 2016, with sampling from 120
communities taking place in January 2016. Survey preparation and implementation has taken a long time,
including a long delay for a permit from the Rwandan government to allow blood sampling.
FINAL REPORT EXPECTED IN DECEMBER 2016
2. Shortening the hungry season through NERICA in Sierra Leone (IPA, MIT, Sierra Leone Agr Res Inst)
This study investigates the impact of early maturing NERICA rice on consumption and nutrition outcomes of
farming communities in Sierra Leone. Most agricultural communities in Africa experience large seasonal
variations in the price of crops. High prices and low stocks of staple crops prior to the new harvest create a
OBJECTIVE 3: Assessing the full range of impacts from CGIAR research (IMPACTS)

of 42
“hungry season” when households reduce food intake with potentially important health and productivity
impacts. All else equal, the worse the base level of nutrition, the more damaging a prolonged reduction in food
intake is likely to be. In the previous phase of this project, high yielding rice (NERICA-3 and ROK16), of which
NERICA rice is also early maturing, were allocated to four treatment arms with varying subsidies. Endline results
from 2013 showed that NERICA treatment households harvested up to 5 weeks earlier and purchased less
imported rice. A survey will be administered to a subset of this sample to estimate the impact of early NERICA-
3 rice harvest on consumption and health at different points in the year.
Progress: This project has been granted a one-year no-cost extension owing to disruption caused by the Ebola
outbreak in the country in 2014. The final report is now expected at end of December 2016. Early results show
that children in households in that received NERICA (either for free or at 50% or 100% of market price) and
agronomic training on how to grow it, see positive effects using anthropometric measures that persist up to
the beginning of the next hungry season. The coefficients for the same measures for the group that did not
receive training but did have access to NERICA at the same fractions of market price are positive, but not
statistically significant and much smaller than those on the treated and trained group at the end of the hungry
season. Previous studies by the same authors have shown that NERICA is susceptible to crop failure when not
grown under correct agronomic conditions, and these findings would suggest that farmer training may be a
necessary condition for achieving certain development outcomes with NERICA.
3. Crop diversification for food and nutrition security in Malawi and Ethiopia (CIMMYT, Lilongwe University,
Georg-August-University of Gottingen, Ethiopian Institute for Agricultural Research)
Crop diversification (CD) is advocated as an essential component for making agricultural systems more
sustainable and remunerative. However, the role of CD on food and nutrition security for smallholders in SSA
has not been rigorously examined. In principle, CD involves cultivation of more than one crop and/or variety
belonging to the same or different species in space and time; to achieve higher spatial and temporal
biodiversity on the farm. The objective of this project is to assess the ex-post impact of CD options in
combination with improved maize varieties on food and nutrition under different social and market conditions.
It aims to estimate the impacts of various types of CD (e.g., legume-maize rotations & legume-maize
intercropping) on nutrition indicators such as calorie and protein consumption, food diversity, micronutrient
consumption (especially iron, zinc, and vitamin A), and childhood anthropometrics. These objectives will be
achieved through the analysis of panel data collected in 2010 and 2014 from 4,000 farm households in Ethiopia
(2,300), and data for Malawi (1,700 households) from 39 and 16 districts, respectively.
Progress: The draft report received in July 2016 for Ethiopia that has evidence that the joint adoption of crop
diversification and modern varieties has higher impacts on per capita calorie, protein and iron consumption
and diet diversity as well as child stunting than adopting each practice in isolation. This result was not
particularly unexpected but there was previously little empirical evidence to support claims that inter-cropping
could have this range of impacts. The results also suggest that adoption of combination of CD and modern
seeds has higher impacts than adopting each practice in isolation. In the case of Malawi, farm species diversity
is positively associated with dietary diversity but the effects are small. Access to markets (for buying food,
selling produce and chemical fertilizers) was shown to be more important for nutrition quality than diverse
farm production.
FINAL REPORTS ARE EXPECTED END OF JULY 2016
4. Looking beyond income: impact of dairy hubs on nutrition in Tanzania (ILRI, Emory U., Tanzania NARS)
This study aims to assess the relationship between farmers’ participation in dairy business hubs and human
nutrition in Tanzania in the context of the dairy value chain More Milk in Tanzania (MoreMilkiT) project, linked
to the CRP Livestock and Fish. ILRI and its partners are working in two regions on designing and pilot testing
dairy business hubs as a mechanism to increase smallholders’ dairy productivity and income. While the existing
project will monitor annually changes in farmers’ uptake of technologies, milk productivity and dairy income,
a plan to monitor change in household livelihood indicators like human nutrition is lacking. Still, prior studies
have shown that an increase in milk production and dairy income at the farm level need not translate into
increased consumption of milk and overall better nutrition for dairy farmers. The proposed study aims

of 42
therefore to complement the MoreMilkiT project and to assess change in nutrition linked with changes in
productivity and dairy income brought about by farmers’ use of the dairy business hubs
Progress: The progress report received in January 2016 showed that this study has moved in a different
direction than expected. The team are attempting to explain variation in nutritional status of household
members for a sample of 373 households. SPIA sent detailed comments on the progress report to the research
team led by Isabelle Baltenweck, to which they responded. The explanatory variable of interest is participation
in a dairy hub, of which the researchers are hoping to identify the causal effect using a combination of
endogenous switching regression (SPIA has some concerns that there could be identification from functional
form) and an instrumental variable (which has yet to be determined from a number of potential candidates).
The outcomes measures for the comparison are dietary diversity for women and children, and total household
food expenditure. Econometric analysis is currently in progress, with a draft of the final report expected in later
in 2016.
FINAL REPORT EXPECTED IN SEPTEMBER 2016
5. Nutritional impacts of irrigated horticulture in Senegal (Columbia., George Washington U., MDG Center)
Evidence is beginning to emerge about the pathways through which intensification of irrigated horticultural
production can affect nutritional outcomes, but most of this evidence is focused on small-scale, home garden
type interventions, rather than larger-scale, commercial and technologically advanced production. This study
will provide new, experimental evidence on the nutrition and food security impacts of an ambitious irrigation
expansion initiative in the Western Sahel, and on the pathways through which these impacts occur. It leverages
a large, funded, randomized controlled trial that evaluates the impact of the PAPSEN-TIPA project in Senegal
which works with groups of smallholder farmers, mostly women. PAPSEN-TIPA disseminates improved
horticultural technologies and equipment based on past and recent agricultural research, including
adaptations of drip irrigation technologies co-developed by ICRISAT and complementary vegetable seeds and
cultivation practices. Previous agricultural studies predict the interventions will generate large impacts on
horticultural and cereal production, women’s income, labor and time use (associated with water delivery to
plots). These are all important potential pathways by which nutrition outcomes can be affected, and the survey
data will shed new light on their relative importance. The broad research question addressed by this study
is: when do agricultural productivity interventions also result in nutritional improvements, particularly for
children? Specifically for this case study, the key questions are “Does the intervention improve diets? On its
own, or only when coupled with nutritional communication (via mobile phones)?” “What are the effective
mechanisms at work, i.e., income, diverse production, time use, etc.?”
Progress: This project got off to a slow start and got underway only in early 2015, but the December 2015
progress report showed that the project was back on track. In June-July 2015 the survey instrument was piloted
in four villages in Senegal. The pilot data indicated that overall, dietary diversity was low among infants and
young children as well as their mothers. All the study villages were visited in September-October 2015 to
ascertain a list of all the households in order to allow the study team to randomly select households that had
a woman with a child between 6-23 months at baseline. The pre-baseline survey was thus essential in order to
allow efficient random sampling of the target population. During the pilot data collection, six focus groups
were conducted to help inform the development of the nutrition education intervention. The topics discussed
in the focus groups included food production, food procurement, infant and young child feeding practices,
seasonal variation and sources of nutrition information. The focus groups have helped identify some potential
barriers and enablers for the nutrition education intervention and will be combined with findings from the
baseline data collection to identify the key infant and young child feeding messages that need to be targeted
in the nutrition education intervention. Baseline survey results related to horticulture cultivated areas, income,
food consumption, food security, anemia, women empowerment, women time use, and diet diversity for both
control and treated are now available.
UPDATE EXPECTED AT THE END OF JUNE 2017
Activity 3.1. Long-term / large-scale impact assessment studies
The basic idea behind this work is to generate studies that credibly document the impacts of successful CGIAR

of 42
research adopted at scale and over the long term using best available methods. Estimating the direct and
indirect impacts from widely adopted technologies and policies is of special relevance to CGIAR donors and
other stakeholders, particularly in a climate of high accountability and expectation of linkages between
agricultural research investments and socially desirable outcomes.
While experimental and quasi-experimental approaches potentially have much to offer in terms of rigorous
estimation of causal effects during early stages of adoption and at limited scales within producer populations,
other methods, often less quantitative and seemingly less rigorous but more comprehensive, are needed to
estimate impact over longer time periods and larger spatial scales. In addition to measuring the effects on crop
yields and total farm income (or nutritional improvements) of adopters, estimating the impact of widespread
technological change requires consideration of effects on other groups. Widespread technological change
often generates significant partial and general equilibrium effects on farm product prices and farm production
resources, especially labor, but potentially land and other inputs that in turn have significant impacts on
poverty, nutrition and other welfare measures affecting adopting farmers as well as other populations. Indeed,
in many cases, it is believed these widespread indirect effects dwarf direct impacts in the adopting regions.
The usual impact studies, which estimate producer and consumer surplus, take the first step of including
effects on consumers of the product whose production efficiency has improved, and such studies undoubtedly
have shortcomings that should be addressed. But in addition, they often do not in any way consider the indirect
effects on farm input markets or on markets of production complements or substitutes. To what extent it is
possible to demonstrate direct and indirect causal linkages from CGIAR-related technologies in these fairly
complex pathways remains to be seen, but this is the goal of this activity.
In early September 2014, SPIA issued a call for expressions of interest to fund studies that seek to measure the
impacts of widely-adopted CGIAR research related innovations. Seven studies were funded out of the 12 full
proposals received (8 impact + 4 adoption studies) in January 2015. An inception workshop for the set of
studies selected was held in July 2015 at IFPRI, DC. The workshop had two objectives: (1) to provide specific
feedback on technical and operational aspects of the funded studies, and (2) to provide an opportunity for
participants to exchange views on the operational and data-related aspects of long-term, large-scale studies
of CGIAR research impact studies – for instance, lessons from DNA fingerprinting work to estimate varietal
diffusion; reflections on using micro data for macro analysis; and challenges in sampling and extrapolation for
such studies.
The seven funded studies are as follows:
1. Adoption and diffusion of C88 potato variety in China: Spatial variability of productivity gains and cost
savings and value chain development (CIP, Virginia Tech, and Yunnan Normal Univ)
Potato variety Cooperation 88 (C88) is among CIP’s biggest single varietal successes to date. In 2010, C88 was
estimated to be grown on about 400,000 ha, with economic benefits estimated at US $350 million annually.
But there are questions about the validity of adoption estimates for the variety that were based on non-
structured expert elicitation methods. The objectives of the study are to:
A. Obtain rigorous estimates of the adoption of potato varieties in Yunnan province
B. Analyze the impact of C88 by comparing yields and costs relative to the varieties it replaced, and,
C. Estimate market-level impacts of C88 by examining benefits along the potato value chain.
Progress: Household and community surveys, DNA fingerprinting exercises and interviews with various actors
along the potato value chain (mainly potato chip producers) to understand qualitatively the value chain, have
all been completed. Economic analysis is on-going. A surprising result (thus far) is the observation of some dis-
adoption and degeneration of C88, which means that the variety’s impact has likely plateaued in Yunnan. The
project is on track and over the next four months the study team will be finalizing estimates of the economic
impacts of adoption of C88 in Yunnan province and completing their qualitative assessment of the value chain
and writing up results.
FINAL REPORT EXPECTED IN NOVEMBER 2016

of 42
2. Estimating improved Tilapia adoption using DNA fingerprinting: Philippines and Bangladesh (WorldFish)
The study is being undertaken to update (and improve on) estimates of adoption in the Philippines and
Bangladesh of an improved tilapia strain (GIFT) developed by WorldFish by using innovative tracking of
fingerling diffusion. Specifically, it aims to characterize nucleus populations of key GIFT and non-GIFT tilapia
strains using genome-wide SNP genotyping approaches, and to validate (or otherwise) recently completed
field-based adoption estimates.
Progress: Successful collection of tissue samples from nucleus populations of all the major tilapia strains
identified during the inception meetings has been achieved. The strains sampled are: GIFT-Malaysia, GIFT-
Philippines, FaST, GET-ExCEL, Chitralada and BEST, which account for the majority of tilapia production in
Philippines and Bangladesh. Genomic marker development has identified over 13,000 SNP markers, which will
be used to characterize the nucleus populations and for testing the genetic origin of hatchery samples.
Currently, hatchery tissue collections, and the compilation of government production records are underway
and expected to be completed soon. The project is on track.
3. Adoption of improved lentil varieties in Bangladesh: Comparison between expert estimates, nationally
representative farm household survey and DNA fingerprinting (ICARDA and Virginia Tech)
The overall objective is to document current lentil variety adoption levels and identify the determinants of
adoption and trait preferences of lentil farmers in western Bangladesh. In particular, the study compares lentil
varietal adoption estimates obtained by expert opinion with those obtained by household surveys and
compares those with more reliable estimates generated from DNA fingerprinting.
Progress: Data collection, digitization and cleaning for 1000 households across 10 districts in western
Bangladesh completed. Village level data for 52 sample villages has been collected and is being digitized, and
samples have been taken and analysis is underway for DNA fingerprinting exercise. The project is on track. An
early result shows the total area under lentil in these 10 districts alone is estimated to be well over 250,000 ha
of which about 80% is under improved varieties (subject to DNA validation), whereas official statistics had the
total lentil area (in 2010/11) at 70,000 ha and was expected to have grown by 10% each year to reach 100,000
ha for the same period.
4. A systematic and global assessment of the impact of CG technologies on poverty (IFPRI and World Bank)
The study will provide a systematic and global assessment of the overall impact of CGIAR research on growth,
poverty, food security and environmental indicators by combining large-scale global multi-sectoral dynamic
computable general equilibrium and households modelling using an outmatched dataset of household surveys
covering 80 percent of global poor. The framework is supported by researchers used to using such tools and
data to capture the full payoff of CGIAR research both in terms of macroeconomic and microeconomic, direct
and indirect effects.
Progress: The primary focus to-date has been on assessment of productivity implications. Three
complementary but highly different approaches have been/are being used: (i) a review of the evidence
available in the existing literature; (ii) back-casting approach to assess productivity growth in key commodities,
and; (iii) a Delphi approach to elicit opinions from experts—informed by evidence from 1 and 2. Progress is
underway on all three approaches. The study leaders have adapted an enhanced MIRAGRODEP model to be
able to perform the back-casting exercise at the macroeconomic and sectoral level. MIRAGRODEP is a recursive
dynamic multi-region, multi-sector model CGE model. International economic linkages are captured through
international trade in goods and foreign direct investment. The project is on track.
FINAL REPORT EXPECTED IN JANUARY 2017
5. Using global agricultural, health and demographic datasets to identify the impacts of CGIAR’s modern
seed varieties since 1960s (UC San Diego and George Washington University)
The objective of this study is to undertake a comprehensive global assessment of the economic, demographic

of 42
and health impacts of MV releases by integrating hundreds of spatially precise Demographic and Health Survey
(DHS) samples from around the world with detailed information on the timing of MV releases and high-
resolution geospatial data on crop distribution. The study proposes to estimate the impacts on the following
outcomes:
 Agriculture: Yields, area planted, and use of agronomic inputs (fertilizers, irrigation)
 Demography: Birth rates, sex ratios, infant and child mortality
 Nutrition and Food Security: Anthropometrics (birth weight, weight-for-height and height-for-age);
proxies of nutritional status (blood haemoglobin, night blindness as a proxy of Vitamin A availability);
direct indicators of food intake (available across a range of food groups)
 Economic Indicators: Wealth proxies (including asset indices), school enrolment and attainment, and
use of health care. Night light data (post 1992) used as spatial indicator of economic productivity.
Progress: After harmonization and integration of various data sets, initial econometric analysis of MV diffusion
effects on agricultural (yields) and demographic and health outcomes in 18,000 rural villages across 37
developing countries was completed. A report is being prepared summarizing all of the analysis using Evenson-
Gollin MV diffusion data and DHS outcomes, and a research paper analysing the impact of MV diffusion on
infant mortality (the most robust empirical result) is also being prepared. The use of GAEZ data to exploit
biophysical conditions for improved empirical identification of MV diffusion effects is being explored. A
country-level case study of impact of rice MVs in Cambodia is also planned.
6. Measuring the impact of IFPRI’s research on Strengthening Food Policy through Intra-Household Analysis
on the behavior of international NGOs (TANGO)
The overall objective of this study is to assess the extent to which findings of the IFPRI research program on
Strengthening Food Policy through Intra-Household Analysis have been widely adopted at policy level and, in
particular, evaluate the extent to which research findings and policy guidance on use of intra-household gender
analysis have been operationalized in smallholder agriculture programs, projects and components funded by
OECD-DAC members and implemented by INGOs.
Progress: Three main tasks identified in the proposal relate to: the critical junctures analysis; gender policy
analysis of DAC members; and agricultural project documents review for 4 selected least-developed countries.
At its launch workshop in October 2015, the study team agreed on a detailed work plan for implementing these
tasks and decided to hold a second workshop in February 2016 to review results, approve the mid-term
progress report, and plan the implementation of Phase Two. The first task included a bibliographical review, a
citation search and a documentary network analysis of the women in agriculture literature since 1994, as well
as semi-structured interviews with key contributors to the IFPRI research program. These are completed. The
second task involved a number of discrete activities including: an Inventory of gender policy documents of DAC
donors; a key word search; qualitative analysis of gender documents; structured Interviews, and identification
of pathways through which research results pass from originator to donor. Most of these are completed. The
third task involves assembling documentation for projects retained for in-depth study but has met with little
initial success (due to limited response of gender experts for phone interviews in the DAC donor HQ to facilitate
access to this documentation). This task will now be shifted to the collection and analysis of project documents
in Phase 2. Phase 1 is therefore completed (or soon will be) and the team is now moving on to Phase 2, the in-
country fieldwork survey and analysis.
FINAL REPORT EXPECTED IN AUGUST 2016
7. Assessing the impacts of improved cassava varieties in Nigeria (IITA)
The study aims to document the extent of adoption of improved varieties of cassava in Nigeria – as a group
and for individual varieties, identify the determinants of uptake and spread of these improved varieties, and
estimate the causal effect of adoption of improved cassava varieties on crop yields, incomes, food security,
and poverty. It also intends to investigate heterogeneity effects focusing on gender differentials in adoption of
improved cassava varieties in Nigeria.

of 42
Progress: 2500 randomly selected cassava growing households have been surveyed across the four major
geopolitical regions of Nigeria based on an earlier sampling frame. Data collection, which included a wide range
of information related to adoption of varieties, plot level input-output information, and household
consumption and other outcome variables, started in 2015 and was completed early in 2016. The household
data indicates high variability in adoption across the regions, from 79% to 30%. A total of 7428 different leaf
samples were taken for DNA finger printing. Extraction of DNA from all samples was completed, sent to Cornell
University for genotyping by sequencing and based on those results, identification of improved cassava
varieties has been completed by IITA. Data coding, cleaning and analysis has been done in STATA. Analysis is
now underway. Comparisons between DNA fingerprinting and self-perception based adoption estimates will
be made. The impact of adoption across a range of social and economic outcomes will be assessed. Project is
on track.
FINAL REPORT EXPECTED IN JUNE 2017
All but one of these studies will be completed by January 2017. A workshop reviewing progress with SIAC
Activity 3.1 (long-term large scale) studies is scheduled for 30 July 2016 in Boston, immediately prior to the
AAEA meetings. Information about the workshop is available from the SPIA website.
Activity 3.2. Micro-scale impact studies using experimental and quasi-experimental methods
There are two parts to this activity led by Karen Macours at the Paris School of Economics: three studies were
identified for pilot funding in 2013; and in 2014, a two-stage competitive call was launched (details below).
The three studies identified for pilot funding focused on understanding adoption and its relationship with
subsequent impacts of technologies developed by the CGIAR, as below:
1. Study of the impacts of adopting rainwater harvesting techniques on yields and soil quality in Niger
(Tufts, Catholic Relief Services, ICRISAT)
This pilot study aimed at understanding the barriers to adoption of rainwater harvesting technology (demi
lunes), as well as the effect of adoption on soil quality, yield, costs and revenues. Adoption of demi-lunes
technology is low in Niger – less than 10% of the farmers are estimated to use micro-catchments on any part
of their land. This despite research station and on farm trials showing positive impact on soil
erosion/degradation, and reduced risk of crop failure.
The pilot was across 30 villages in the Dosso region of Niger: villages were chosen/targeted because they are
most affected by soil degradation. Within each village, 25 farmers were selected and villages were randomly
assigned to one of three interventions (a) training: basic training for RWH techniques, including the necessary
equipment; (b) non-conditional cash transfer (UCT): training on RWH techniques and an unconditional cash
transfer (cash advance) worth US$ 40, approximately 50% of the labor costs labor and equipment costs
associated with constructing these structures on one hectare of land; and (3) conditional cash transfer (CCT):
training on RWH techniques and a conditional cash transfer, whereby farmers receive a cash payment for every
demi-lune of acceptable quality, similar to the per-RWH value of the unconditional transfer.
Progress: Baseline data collection and training was completed in February 2015. In November 2015, the endline
survey captured impact on soil quality and yields (subjective measures, self-reported by farmers). This data is
being analyzed and results will be available in July/August 2016.
The pilot results on training and adoption found the following:
 Training: The interest in training was high – 85% of the households attended training regardless of
the group (training, UCT, CCT) they were assigned to.
 Adoption: households in the CCT group were less likely to construct demi-lunes on their plots (13
percentage points, significant at 1%). However, the number of demi-lunes constructed per hectare
was the highest in the CCT group i.e., 47 per hectare versus 30 (training group). The technical norm
suggests 250-300 per hectare. So, while cash transfers help ease credit constraints (farmers can hire
labour), CCT seems to discourage such investment.
FINAL REPORT EXPECTED IN SEPTEMBER 2016

of 42
2. Drought Tolerant (DT) seeds alone or bundled with innovative financial technologies? A proposal to
evaluate the impact of bundling DT maize and index insurance to reduce vulnerability in smallholder
African farming (UC Davis, IITA, CIMMYT, Univ. of Georgia)
SPIA funded a planning workshop to determine the way forward on an RCT to evaluate the impact of bundling
DT maize with index insurance by researchers from CIMMYT, IITA, UC Davis and Univ. of Georgia. Considering
that DT maize does not perform well under extreme drought conditions (~20% of the time), bundling an
insurance product that covers extreme events might lower insurance costs and deepen impact of DT
technologies.
Progress: The outcome of the workshop was a full proposal submitted to a number of donors including USAID
and CCAFS. USAID funded the study (US$ 2.2 million) in 2015, and the project will run through 2019. Outcome
variables to be measured include adoption of DT seeds, agricultural investment (area planted, investment in
yield) as well farm yields and real incomes (especially in years of drought). The study will also document nutrition
and human development impact indicators (dietary diversity, food security).
WORKSHOP REPORT SUBMITTED
3. Learning regarding ISFM through farmer’s own experimentation during researcher-managed on-farm
trials (Paris School of Economics, IITA)
This study sheds light on the biases in agronomic trials by analyzing how different variations of inputs in trials
affect the input allocation in differential ways, by analysing (a) whether farmers randomly selected to
participate have different results than farmers chosen by researchers, (2) whether participating farmers
themselves adopt ex-post, and (c) longer term impacts. This is important because it is assumed that profitable
technologies produced by the CGIAR are not adopted because of other binding constraints, but the evidence
on profitability comes from research station or researcher-managed on farm trials, and hence may not be
relevant for real world conditions.
Progress: The study commenced in November 2013 with identification of farmers interested in experimenting
with new commercial products, and collection of socio-economic baseline data. 96 villages were assigned to
two treatment and one control group, with 10 interested farmers in each village participating in the study
across three seasons, offering an opportunity to understand how learning occurs. Standard agronomic
protocols were followed wherein a scientist worked with each farmer and in each season in the treatment
groups to implement an experimental trial on their land in which a random control plot be compared with 5
different randomly selected treatment plots. The treatments on each of the plots varies, as does the
combination of treatments to be tested by each participating farmer. Hence, in a given season, a farmer tests
different combinations of seeds (soy, maize, soy-maize intercrop) and fertilizer packages. Through August-
September 2014, follow-up surveys were conducted, and in December 2014, qualitative data collection took
place during the second season. Throughout agricultural seasons, monitoring of the trials continued, providing
data on compliance with the experimental design, issues with pests, weather damages, and other confounding
factors.
Take-up during the second season was much lower than the first – a high proportion of farmers (18%) selected
for treatment did not participate for various reasons that included a lack of interest in crop cultivation during a
short rainy season or miscommunication on land preparation between agronomists and farmers. Baseline data
confirmed that community selected farmers for experimental trials are significantly different from randomly
selected farmers – they have higher skill levels on all dimensions. While there were some difference in
agricultural practices, this was mixed – significant differences for some practices (use of manure, fertilizer,
hybrid seeds etc.) and no significant differences for others. At the end of the first season, while community
selected farmers appeared to be more productive (control subplot production was higher compared to the
same for randomly selected ones), the returns to treatment was not significantly different. And while all types
of farmers adopted some of the practices (no significant difference between community and random),
community selected farmers were more likely to have updated their beliefs about returns to inputs for maize
production. The analysis of the future rounds of survey will shed more light on these fundamental questions in
technology adoption.
The 2014 two-stage call was launched in mid-2014 with the objective of furthering our understanding of how

of 42
Randomized Control Trials (RCTs) can contribute to an understanding of specific causal pathways from
technology adoption. The 2014 studies addressed two of the following:
• convincingly show the effect of adoption of a CGIAR technology on productivity (downward shift in
marginal cost of production or upward shift of production function)
• help understand the linkages/mechanisms from adoption to impact (on income /nutrition / good or
bad environmental outcomes)
• help understand other “deeper” questions that limit the effectiveness (sustained adoptability) of
current CGIAR research – understanding the technology production process
• address academically interesting questions related to behavioural/welfare/development economics
etc.
Three studies were contracted between November 2014 and January 2015, and an inception workshop for the
studies was held at MIT, Cambridge MA, in February 2015. At the inception workshop, researchers invited to
submit full proposals but not funded benefitted from discussions on the funded studies, and on the second day,
received detailed feedback from other researchers (Alain de Janvry, Karen Macours, Tavneet Suri, MIT PhD
Econ students, SPIA) to improve their proposals.
1. A multiple intervention approach to increasing technology adoption with a view towards scaling-up:
Evidence from Mexico (QFD Mexico, UC Berkeley, ITAM, World Bank)
This study of how improved recommendations (based on soil analysis) and grants in combination with other
extension services affects technology adoption (fertilizer, precision drill) will contribute to a better
understanding of heterogeneity-adoption links. Heterogeneity is one among the many plausible explanations
for low adoption, missing markets for credit, information, etc. being another.
As a part of the experiment, soil analysis is used to make input recommendations on fertilizer dosage and timing
of application with additional information on precision sowing (using a drill) and timing of herbicide use. Studies
have suggested that soil quality is central to understanding fertilizer adoption levels, and in the study area,
farmers have been found to overuse urea but use much less KCl and no micro-nutrients. Besides providing
recommendations in a document and explaining this to farmers, the experiment also provides information on
prices, quantities and total costs for each recommended input; the geo-location of agro-dealer for fertilizer;
and aggregates demand and coordinates logistics for hiring precision sowing drills. In-kind grant of US$ 150
worth of inputs (half of average per-hectare cost), an inflexible grant restricted to items on the
recommendation list, or a flexible grant (any inputs can be purchased) is experimentally varied.
Progress: Work on the SPIA-funded aspects of this project began in January 2015 with a baseline, and 981 maize
farmers were recruited for the study in February 2015. Between February and April 2015, various interventions
were delivered. Mid-term survey that collected plant density information was completed in July 2015, and a
commercialization survey was done in February/March 2016. By the end of November 2016, results from
various follow-up surveys will be available. The study will also conduct an endline post harvest in November
2016 with a focus on welfare impacts, and these results should be available by May/June 2017.
Preliminary results from follow-up surveys indicate that take-up rates are significantly higher (not very
surprisingly, grants matter) for farmers who received individualized recommendations and inflexible grants
(T1), averaged recommendations and inflexible grant (T2), and averaged recommendations and flexible grant
(T3) along with extension - compared to control (C) and farmers who received averaged recommendations and
agricultural extension (T4). However, the take up of fertilizer and precision drill was similar regardless of the
flexibility in grant i.e., similar rates for T2 and T3. Curiously, T1 (individualized+inflexible grant) take-up is lower
than T2 (averaged+inflexible grant). Farmers’ certainty about soil quality increased in all treatments that
received recommendations (T1 through T3), and their expectations of yield volatility was lower. And finally,
whether recommendations were individualized or averaged did not seem to matter for plant density.
PROGRESS REPORT EXPECTED IN DECEMBER 2016
2. Drought resistance and water saving in rice production in Bangladesh (UC Berkeley, Tufts, IRRI)
This study of drought-tolerant, shorter duration rice (BD56) in Bangladesh will examine the impact on
productivity, water savings and the local market for water, and welfare impacts. Much like other studies in this

of 42
SIAC activity, it will examine the heterogeneity of impacts (large versus average farmers), and also, how
diffusion occurs, by randomizing who the demonstrator farmers are. Such a variation (in demonstrators) is
motivated by an interest in improving agricultural extension, and this study is only one of the three experiments
along similar lines in India and Bangladesh by the same team.
Progress: Work on this project began in May 2015 as a pilot experiment across 35 villages, instead of a full-
fledged RCT because there was a large-scale flooding that destroyed crops and made census difficult. During
the pilot, 5kgs of seed was provided to 5 farmers in each treatment village. Demonstrators were of five types.
That is, they were (1) either randomly chosen, (2) the largest farmers, (3) picked on the basis of self-reported
willingness to pay (WTP), (4) voted by peers, or (5) closest to the average farmer (village mean variables). Pilot
data analysis confirms that BD56 offers a shorter duration, and affects the crop choice for dry season. That is,
farmers who grow BD54 are more likely to plant wheat and pulses – increasing the number of cropping seasons
from two to three in an agricultural year, compared to control. BD56 was also shown to produce reduced yields
compared to longer duration varieties, but this varied by the type of demonstrator farmer. The pilot also
showed that larger farmers spread information better.
For the full experiment, a census survey to select demonstrator farmers as well as the baseline survey was
completed in April and May 2016. In 192 villages, 5 demonstrators have received 5kg of BD56 (about 35% of
wet season rice area), and demonstrators are either (1) chosen randomly, (2) the largest rice farmers, or (3)
were selected by an agricultural officer. Hypothetical (comparable) recipients selected by the same selection
mechanism in 64 control villages have received BD51 (Swarna Sub1) to enable an assessment of impact. The
first follow-up survey on production and welfare outcomes will occur in February 2017, and the study will
progress independent of SIAC funds beyond June 2017 to track diffusion (August 2017 and August 2018) and
complete a second follow-up survey in February 2018.
FINAL REPORT EXPECTED IN APRIL/MAY 2017
3. Social networks to promote new agricultural technologies in Nepal (Yale, ICIMOD)
The study, drawing on previous work in Malawi (paper), examines how social learning can improve agricultural
extension and increase the adoption of new technologies (in this case, maize intercropping in Nepal).
Intercropping adoption rates are below 10% in Nepal, and pilot data indicates people are not aware of the
potential to intercrop maize with tomatoes, ginger, or French beans. Hence, an information intervention could
increase adoption rates. At the same time, agricultural extension services face challenges in promoting new
technologies, and there may be fairly inexpensive ways to leverage social learning to improve extension.
Following the Malawi experiment where it was found that peer and lead farmers did no better than pure control
but incentivizing them had a positive impact on adoption (of pit planting), this study experiments with the type
of incentives (flat versus performance based). Besides impact data on yields and welfare (nutrition), this study
will contribute to our understanding of the heterogeneity of impacts: there will be differential adoption by
farmers from varying extension/information delivery mechanisms, allowing such an analysis of data. SPIA
funding is for additional survey rounds of the existing experiment which began in 2014.
Progress: This project experienced significant delays due to the Nepal earthquake in April 2015 (two of the ten
districts were severely affected), and a fuel crisis between September 2015 and February 2016. Hence, the
timeline is being extended by a full year. That is, while the fourth survey round and related data
cleaning/analysis ends November 2016, the contract will be extended so SPIA can have an intermediate output
by April 2017 with results, and a final report in July 2017 (after the program ends).
Analysis of baseline data indicates that the lead and peer farmers have been chosen appropriately: lead farmers
(community leaders and early adopters) and peer farmers (well-connected and seen as leaders but similar to
the average farmer) are wealthier than others in the sample; they tend to own more land and get more
extension visits in a year; and, they are more aware of maize intercropping. After one year of intervention, data
shows that knowledge of maize intercropping has increased across all ‘treatment’ groups. Take-up data
(adoption) is still being analyzed.
PROGRESS REPORT EXPECTED IN DECEMBER 2016
A mid-term workshop for the studies was held at FERDI, Clermont-Ferrand, France in June 2016, and included

of 42
a discussion on lessons from the research implementation and early results, and the implications for SIAC
Phase-II. This half-a-day workshop followed a larger ‘Learning for Adoption’ (2-day) workshop, coordinated by
Karen Macours (PSE and SPIA), Alain de Janvry and Elisabeth Sadoulet (UC Berkeley), and Vianney Dequiedt
(CERDI), that aimed to explore research avenues and innovative designs for farmer learning for adoption.
Activity 3.3. Under-evaluated areas of CGIAR research
Many studies over the years have sought to document the impacts of agricultural research although the vast
majority of these have focused on crop germplasm improvement, i.e., adoption and impact of improved crop
varieties. As such there remain serious gaps in the extent to which impact assessment of other components of
the CGIAR portfolio have been conducted. To fill this gap, one of the activities of the SIAC program targets what
we consider to be under-evaluated areas of CGIAR research, such as livestock management, irrigation and
water management, agroforestry, policy and social sciences, biodiversity and natural research management.
There are two parts to this activity: (1) as an initial step in the SIAC work on under-evaluated areas of CGIAR
research, commissioning of critical reviews of the impact assessment work-to-date on specific research areas
(irrigation, livestock, agroforestry etc.); and (2) a wider call for ex-post impact assessments to document the
adoption and the direct impacts of CGIAR research (e.g. reduced water usage for irrigation or improved yield),
and where possible, poverty, social, and environmental impacts of these innovations.
As an initial step, SPIA, in April 2014 commissioned Doug Merrey, a consultant with considerable experience in
water management research, to conduct a critical review of the impact assessment work to-date on irrigation
and water management research. This desk study includes IA work done within and outside the CGIAR, and
evaluates how comprehensively and effectively these assessments cover the field of irrigation and water
management research since 1990. Merrey’s report concludes that while CGIAR has invested an estimated US$
800 million (nominal, not constant dollars) in irrigation and water management research over the past 25 years,
it has substantially underinvested in economic, social and environmental impact assessments of this work. Only
about 14 cases (out of 25 available) qualified as IAs of uptake, outcomes, and impacts of irrigation and water
management research. There is no definitive answer to the question on overall influence, outcomes, and
impacts of CGIAR water management research since 1991, and there is too much missing data to make credible
claims, even as there is some evidence of positive impacts by individual projects/programs on broader scientific
literature, policy and research agendas, and food security/incomes of farmers. The review identifies the major
constraints and limitations, e.g., methodological, data-related, resource-related, etc., of previous studies and
offers guidance regarding some specific candidates for IA studies of CGIAR research on irrigation and water
management that have good potential for documentation.
The review of the impact assessment work on livestock and livestock-related research in the CGIAR is the second
scoping study commissioned by SPIA to assess the impact evidence across these under-evaluated areas within
the CGIAR portfolio. In March 2015, two consultants, Sam Jutzi and Karl Rich were commissioned to evaluate
the extent and quality of ex-post impact assessment activity on livestock related research in the CGIAR to-date.
The report was internally reviewed by SPIA, and finalized at the end of November 2015 with a foreword by SPIA
Chair. The final report was published in May 2016 along with a SPIA commentary. As was the case in the
previous scoping study, the literature does not provide strong evidence of impact from the CGIAR’s past
investments in livestock and livestock-related research. Of the twelve studies that form the basis of the report,
none offers evidence that would justify, in aggregate, the investment in livestock research in the CGIAR –
estimated to be about US$ 1 billion (unadjusted for inflation). That does not mean that there has been no
impact, nor does it imply that investments in animal agriculture have been unjustified. But it is clear there are
no credible studies showing long-term or large-scale adoption of CGIAR-derived innovations or policies from
livestock research. At the same time, it is important to note the existence of a larger set of micro-level case
studies that have examined the link between the adoption of livestock-related technologies /management
practices and their direct farm-level impacts. Some of those may offer useful analysis of impacts at the farm
level. These too, in SPIA’s view, merit serious attention in terms of their potential for adding to the evidence
base, understanding constraints to adoption and impact, and feeding a learning agenda. These qualifiers are
not intended to detract from the thoroughness of this review or the importance of the main findings and
recommendations. On the contrary, the substantial gaps identified by this and the irrigation and water

of 42
management review underline the need for building a stronger and more rigorous evidence base in examining
the impact of these under-evaluated areas of CGIAR research.
Given the dearth of evidence of impact of CGIAR research outputs in major areas of CGIAR investment, in June
2015, SPIA issued a two-stage call for proposals for impact assessment studies of under-evaluated areas of
CGIAR research, particularly inviting studies in areas such as irrigation and water management, livestock
management, agroforestry, biodiversity, social science & policy and NRM. Of the 26 EOIs received and internally
reviewed in August/September 2015, 10 were invited to submit full proposals at the end of October 2015. Four
studies were approved for funding in December 2015, as per below. An inception workshop was held on 10
March 2016 in Washington D.C., immediately prior to the meeting on the future of SIAC and SPIA. Contracts
were finalized for all four studies and work has commenced.
1. Forest co-management in Guinea: a multi-scale, multi-output ex-post impact analysis (Virginia Tech and
CIFOR)
This ex-post impact assessment aims to document the durable environmental and household livelihood
benefits of research on the institutional management of forests under the LAMIL project in Guinea eight years
after the cessation of project activities. The durability, spread, and impact of innovations generated between
2005 and 2008 in project treated forest reserves and adjacent villages will be estimated through a unique two-
level analysis of environmental and household livelihood benefit. A household survey will be conducted in 8
villages adjacent to the project forest reserves and 8 villages adjacent to control forest reserves to document
the durability of project innovations within treated forest reserves and adjacent villages, as well as the spread
of project innovations to control forest reserves and their adjacent villages. Livelihood impacts will be inferred
through the estimation of differences in forest, agroforestry, and agricultural benefit flows associated with
project innovations in adopting households and otherwise similar non-adopting households.
PROGRESS REPORT EXPECTED IN NOVEMBER 2016
2. Adoption and impact of Alternate Wetting and Drying (AWD) water management for irrigated rice in the
Philippines (North Carolina State University, IRRI and NIA)
IRRI and national agricultural research and extension system (NARES) partners have worked together to
develop and promote the AWD water management approach for a number of rice-producing countries in Asia.
This study aims to comprehensively determine the multi-dimensional impact of AWD in the Philippines, i.e.,
document the economic, poverty, socio-cultural, gender, and environmental impacts of AWD, as well as
providing an estimate of the rate of return on the research investments made for the development of this
technology. State-of-the-art quantitative impact assessment techniques (i.e., combined DID/ randomization)
as well as qualitative approaches (i.e., network/contribution analysis) will be used to achieve project objectives,
and will address: (1) selection issues and confounding factors, (2) impact heterogeneity (i.e., upstream vs
downstream), and (3) poverty, socio-cultural, and environmental impacts.
3. Assessing the adoption and economic and environmental impacts of Brachiaria grass forage cultivars in
Latin America focusing on the experience in Colombia (CIAT, Michigan State University, Universidad de
los Andes-Colombia, and CORPOICA)
Brachiaria grasses were introduced to Latin America from Africa in the early 1950’s and has reached a large
adoption in Brazil. CIAT and other partners have led research and agronomic evaluation that has expanded the
use of Brachiaria cultivars to other countries in Latin America. Some project reports and statistics on seed sales
suggest a large adoption of Brachiaria grasses in these countries but there are no rigorous studies that have
documented this adoption process. There is also no evidence of the impacts that the adoption of Brachiaria
may have generated. This proposal aims to contribute to filling this gap by rigorously documenting the adoption
and impacts (on livelihood and environment) of Brachiaria cultivars in Colombia using an econometric analysis
of household level data, and by estimating aggregate impacts of Brachiaria in Colombia, Peru, Nicaragua and
Costa Rica using existing data.

of 42
4. Assessing the downstream socioeconomic and land health impacts of agroforestry in Kenya (ICRAF, Vi
Agroforestry, and Univ. of Illinois)
There is a dearth of evidence on the longer‐term impacts of agroforestry practice, despite its widely purported
ability. The proposed study aims to make a significant contribution to addressing this evidence gap by
comparing villages in three distinct locations in western Kenya where a number of integrated agroforestry
practices have been intensively promoted with comparable villages where they have not. To address program
placement and self‐selection bias, propensity score matching (PSM) and spatial econometric methods will be
combined with difference‐in‐differences estimation. Qualitative methods will also be used to triangulate key
findings and interrogate impact pathways, unforeseen outcomes, and unexpected quantitative results. Key
outcomes of interest include household wealth status (as measured by asset ownership and other locally
relevant wealth indicators); Progress Out of Poverty Index; Multidimensional Poverty Index; levels of livelihood
diversification; and various land health metrics constructed through the analysis of satellite imagery, including
soil organic carbon, soil erosion, soil pH, root depth restriction, and fractional vegetative cover.
All four studies are scheduled for completion by the end of June 2017. These studies will provide high-quality
evidence of the impact of some of the more important CGIAR outputs/outcomes that have remained under-
evaluated, and showcase alternate approaches to IAs when traditional experimental or even the more credible
of the quasi-experimental approaches are not feasible.
Activity 3.4. Meta-analyses at System level: various studies
1. A comprehensive empirical assessment of the returns to agricultural R&D worldwide (University of
Minnesota)
Drawing on the newly completed version 3.0 of the InSTePP Returns-to-Research Database (a global
compilation of the returns to research evidence spanning the period 1958 to 2015), the research team led by
Phil Pardey is preparing several themed summaries and undertaking a formal analysis of new data on returns
to agricultural research to develop a series of briefing notes and technical reports which will:
 Summarize the new and comprehensive returns-to-research evidence emphasizing graphical, tabular,
and mapped representations of the data, with associated descriptive text and commentary.
 Undertake a critical review of agricultural R&D returns, investigating on the one hand whether or not
there is a tendency for the returns to investments in R&D to decline over time and on the other hand
analyzing methodologies that will enable the historical internal rate of return evidence to be
recalibrated into modified internal rates of return.
 SPIA received a proposal from the International Science and Technology Practice and Policy (InSTePP)
Center at the University of Minnesota on 1 November to support the above program of work which
extends the work that InSTEPP have already been doing in estimating the economic returns to
agricultural research using meta-analysis techniques. A recommendation to fund the study was put
to the SIAC PSC in late November 2015 and approved. Target date for completion is November 2016.
Progress:
 “Returns to Food and Agricultural R&D Investments Worldwide, 1958-2015”: A complete draft of this
brief has been prepared and formatted, and is ready for posting on line.
 “Returns to Food and Agricultural R&D Investments in Sub-Saharan Africa, 1975-2014”: A complete
draft of this brief has been prepared, and is under review for publication by Food Policy.
 “Returns to Food and Agricultural R&D Investments in Latin America and the Caribbean, 1970-2013”:
Data work completed, the associated figures and tables have been prepared, and drafting has begun.
 “The Returns to CGIAR Research, 1970-2015”: The data are finalized and analysis and write up of this
brief will begin later this summer.
 “Are Agricultural R&D Returns Declining and Development Dependent?”: An initial round of
econometric analysis has been completed and a first draft of the paper has been prepared. The paper

of 42
was accepted as a selected paper for presentation at the Agricultural & Applied Economics
Association (AAEA) annual conference in Boston in early August 2016.
2. Impact of new technology on agricultural productivity – a re-examination (Bob Herdt & Sarah Mine)
There is a long and complicated pathway from adoption of technologies/practices to impacts (as in the CGIAR
System Level Outcomes). SPIA, under the leadership of Bob Herdt and with the assistance of consultant Sarah
Mine, has sought to examine the first and most direct link in this pathway – from adoption of
technology/practice to productivity or other measurable outcomes, with these specific objectives in mind: (a)
to identify the evidence, in the form of impact assessment (IA) studies, (b) to characterize the IAs, and, (c) to
evaluate the quality of the evidence using pre-determined criteria. The study has progressed in two phases.
The first step of this study involved designing, testing, and finalizing the approach to characterizing the IAs, and
identifying and testing criteria that could be used to assess the quality of evidence. SPIA had a list of 214 studies
that were identified (and passed the first screen) in the Loevinsohn et al. review (2013) submitted to DiFD.
Hence, instead of building a database of relevant IA studies afresh, the characterization exercise initially
focused on this set of 214 IAs. The second step involved (a) completing the in-depth quality review of a sub-set
of studies from the list of 214; (b) expanding the database of IA studies that have examined the adoption-
productivity (or other outcome indicator) pathway using snowball searches and references listed in reviews
other than Loevinsohn et al. 2013, and (c) repeating the characterization and quality assessment exercise for
the IA studies that qualified.
Using criteria slightly less restrictive than Loevinsohn et al., the authors identified 34 of the 214 studies that
reported a relationship between technology and agricultural productivity: 20 of the 23 with yield data showed
a positive relationship between use of technology and yield, three were inconclusive; 22 of 27 examining
income showed a positive relationship between technology use and income, the others were inconclusive. They
found 20 additional studies published since 2013. Nine of the ten reporting yield found a positive relationship
and all 14 reporting income found a positive relationship. All of the additional studies used modern analytical
procedures that attempt to recognize and account for the fact that adopting farmers may differ in many ways
from non-adopters, enhancing our confidence in the robustness of the findings.
FINAL REPORT SUBMITTED - TO BE REVIEWED AND PUBLISHED
3. Trends in investments in CGIAR research (Sarah Elven)
This exercise was, in part, motivated by the lack of ready-to-access data on historical investments in the CGIAR
for specific areas of research (as found during the Merrey irrigation review – activity 3.3). A consultant (Sarah
Elven) reviewed annual reports and financial statements to carefully construct estimates of research
investments by output/area of research/region since the CGIAR inception. Such data can feed into returns to
research investment analyses as well as help identify trends in investment over a period of time.
Progress: Between August 2015 and January 2016, a database of investment (between 1972 and 2013) by five
strategic areas as identified by the CGIAR, investments disaggregated by commodities (between 1972 and
2006), and investment by Center (1970 to 2015), and investment by region (between 1986 and 2014) was
constructed. Report summarizing results and analyzing some of the larger trends was completed over January
and February 2016. A more thorough classification by research areas – beyond increasing productivity, saving
biodiversity, protecting the environment, improving policy, strengthening NARS, or commodities (cereals,
legumes, roots and tubers, bananas, livestock, trees, fish, water) will involve significant effort and time (sifting
through documents that may not necessarily be readily available online). Hence, SPIA will review the report
and decide on a direction of this work in the coming months.
FINAL REPORT SUBMITTED (NOT PUBLISHED)

of 42
The CGIAR will benefit from a structured attempt to support the existing capacity and some emerging
collaborations on ex post impact assessment. Information-sharing and regular interaction are important in
enabling the kinds of dialogue that can raise standards of impact assessment in the CGIAR, as well as ensuring
that individuals have the skills that they need to be successful in their work. Activities towards this objective
include a small grants program (activity 4.1); a targeted program of capacity-building using competitive calls for
collaborations with advanced research institutes / universities (activity 4.2); conferences and workshops on
impact assessment (activity 4.3); support for independently reviewing and publishing quality ratings of impact
assessment studies carried out by CRPs and Centers (activity 4.4); maintenance and enhancement of the impact
website (http://impact.cgiar.org) (activity 4.5).
Activity 4.1. Small grants
In 2013, four projects were funded through the small grants program (a total of US$30K) that has been
discontinued since 2014 owing to the heavy administrative burden for small amounts of money. We are
currently evaluating the utility of this mechanism by following up with small grant recipients as an input to our
decision-making for the future. The four projects were:
1. IWMI, electricity and water pump policy in India: evaluation to assess impact of policy change
2. ILRI, pastoral value chains in Senegal: MSc Fellow to develop social sustainability/environmental
sustainability indicators
3. CIMMYT, agri. technology package in Malawi and Ethiopia: applying endogenous regression switching
model to a panel dataset
4. Bioversity, Home Gardens evaluation in Nepal: new approaches to measurement and evaluation of
gender impacts
Activity 4.2. Strengthening IA capacity in the CGIAR through new partnerships
In an effort to enhance the capacity within the CGIAR, at both Center and CRP levels, to conduct highly credible
ex post impact assessments across a range of research areas, SPIA issued a call in August 2013 for concept notes
from any organization or individual interested in undertaking an activity or a set of activities aimed at building
capacity within the CGIAR for conducting ex-post impact assessment of agricultural research. Eleven concept
notes were received of which five were shortlisted and invited to submit full proposals. SPIA reviewed and
ranked the full proposals, and recommended to (and approved by) the PSC, one for funding initially and a second
after re-considering and re-allocating the activity budgets next year. The top ranked proposal was submitted
by Virginia Tech (involving CIP and CIFOR) and the second from ICRISAT involving the University of Illinois.
1. Virginia Tech with CIFOR and CIP
Virginia Tech (George W. Norton, Bradford F. Mills, Catherine Larochelle, Jeff Alwang) has been working with
CIP (Guy Hareau, Willy Pradel) and CIFOR (Daniel Suryadarma; Herry Purnomo) since early 2014 to strengthen
ex post IA activities at those respective Centers, focusing on these specific areas:
 Classification of CIFOR and CIP research according to how IA can be done
 Potential IA methods and approaches for Center research themes
 CRP-specific impact pathways and theories of change, measuring outcomes and impacts
 Impact-related data collection and archiving methods
 Data collection protocols and management systems to meet IA needs; potential IA approaches
 Conduct two pilot IAs in each Center, jointly with IA officers and scientists
 Conduct learning workshops for project participants and other audiences
Progress: Workshops were held at each institution (CIP in May 2014; CIFOR in July 2014) to provide an overview
of the IA strengthening project, present an overview of IA methods and their use, create broad understanding
of data needs for assessment of impacts of the Center portfolio, and engage in consultations about specific
OBJECTIVE 4: Support the development of communities of practice (CAPACITY-BUILDING)

of 42
impact assessment needs. Meetings were held with senior management and scientists to gain understanding of
IA needs. Findings from these activities are being used to classify research according to IA needs and methods.
Center staff were engaged in identifying the most appropriate themes for the pilot IAs.
For CIP, the two pilot IAs are: (a) an evaluation of impacts of Cooperation 88, an important and highly successful
germplasm variety released and widely disseminated in China; and, (b) an assessment of the impacts of the CIP
gene bank. For CIFOR, the two pilots are: (a) an assessment of the impact of the furniture value chain (a project
conducted in Jepara, Indonesia to better articulate small-scale artisanal furniture producers to higher-value
markets); and, (b) an assessment impacts of CIFOR research on carbon sequestration in peat bogs and
mangroves. Virginia Tech has been working with CIP and CIFOR on the pilot impact assessments. While the initial
plan was to hold a final synthesis workshop in November 2015 and submit a formal report by March 2016,
Virginia Tech requested a 10 month no-cost extension. SPIA proposed a revised timeline for reports and
payments and the contract was amended with a final report now due by November 2016.
UPDATE EXPECTED END OF JULY 2016
2. ICRISAT and University of Illinois
This project aims to enhance impact assessment capacity of CGIAR Centers through a workshop, and by
implementing selected case study analyses addressing specific methodological issues using long-term panel
data. More specifically, the activities proposed in this project are expected to:
 raise the standards of impact assessment approaches and protocols, and promote their application
by social and bio-physical scientists at ICRISAT and other CGIAR Centers.
 provide opportunity to pre or post-doctoral fellows to conduct impact assessments through working
on selected case studies under joint supervision from ICRISAT and collaborating partners.
 establish a community of interest for agricultural scientists to engage in discussion and collaboration
on impact assessment activities, and help establish operating procedures and create an environment
for consistent, credible impact evaluation at ICRISAT and other CGIAR consortium members.
Progress: In September 2015, a multi-center workshop on advanced methods in impact assessment was held in
Nairobi (hosted by ICRAF). The objective of the workshop was to raise the standards of impact assessment
approaches and promote their application by social and bio-physical scientists at the CGIAR Centers. The agenda
was based around the range of econometric tools available for impact evaluation. After the workshop,
University of Illinois researchers and SPIA Secretariat met briefly to reflect on the workshop. Judging by
participation rates, the workshop was a success: while University of Illinois/ICRISAT were hoping to attract 25
participants, they received 52 applications (including from NARS partners), and accepted 38 applicants to attend
the workshop representing 10 CGIAR Centers. At the request of participants who strongly felt the need for an
online discussion space, a board was launched in April 2016 (and is moderated by Univ. of Illinois). Despite some
material posted by Univ. of Illinois and SPIA, there has not been any CGIAR activity on the board till July 2016.
Key milestones achieved to-date in this project include:
1. CGIAR workshop on Impact Assessment
 Held in Nairobi Sept 21-26 2015 for 38 attendees from 10 CGIAR centers
2. Develop Common Framework for Impact Assessment
 Developed exercises in workshop to foster inter-center discussion and collaboration
 Established online discussion board http://impact.cgiar.org/boards/
 Host symposia at multiple conferences to discuss issues of identification and impact assessment
3. Strengthen the capacity of CGIAR scientists and scholars
 Awarded three mini-grants to CGIAR scientists
 Made field visits to support implementation of impact evaluations
4. Complete two case studies based on ICRISAT data
 One case study using VDSA/VLS data completed and currently under review
 Second case study using Zimbabwe CA data is in final stages of revision before being submitted
to a journal

of 42
Activity 4.3: Biennial conference on ex post impact assessment results and methods
On July 26th 2014, SPIA organized a workshop entitled “Measuring poverty impacts of agricultural research” as
a pre-conference workshop at the American Applied Economics meetings in Minneapolis. Seventy participants,
approximately one third from the CGIAR and two thirds from academia, participated in full day of presentations
and extended discussion on the potential and limits of the following families of studies: micro-level econometric
studies; model-based approaches; randomized-control trials; meso, macro and cross-country studies. The
invited speakers were of the highest caliber (e.g. Michael Carter, Tavneet Suri, Will Martin, Marc Bellemare,
John Antle, and Julian Alston). The day concluded with a panel discussion on “Reducing rural poverty as a
System-Level Outcome for the CGIAR”. The workshop was very successful in fostering open discussion across
these specialists within the agricultural economics discipline (Poverty impacts workshop, Minneapolis July 2014
report).
Following the workshop, SPIA (Doug Gollin, with research assistance from Lilli Probst) took on the task of
summarizing these findings and reviewing the related literature in a paper for a non-specialist audience. An
outline of this paper (Poverty impacts paper, Gollin Sept 2014) was presented to the meeting of the CGIAR
Independent Science and Partnership Council at the University of Copenhagen in September 2014.
Immediately prior to the poverty impacts workshop, on July 25 2014 in Minneapolis, SPIA hosted a meeting of
the Impact Assessment Focal Points (IAFPs) from across the CGIAR Centers / CRPs. This was a full day of
presentations from each of the 15 focal points, as well as from SPIA on progress with the SIAC program. This
was a valuable opportunity for Center scientists to exchange information on their current impact assessment
projects and to benefit from advice from a number of high-quality resource people that SPIA had arranged to
attend to provide feedback (Julian Alston, Jeff Alwang, George Norton, Greg Traxler, Bob Herdt, JV Meenakshi).
Proceedings and presentations from the IAFP workshop can be found on the Events page on SPIA website.
No IAFP meetings were held in 2015, but SPIA participated in IEA’s Evaluation Community of Practice (ECoP)
meeting in Rome (November 2015) – there is a fair amount of overlap between ECoP members and IAFPs.
SPIA is holding an IAFP meeting in Boston on 29 July 2016, immediately prior to the annual AAEA. There will also
be a mid-term workshop for SIAC Activity 3.1 which IAFPs will also be able to attend. The objective of the IAFP
meeting is to give CRPs an opportunity to present their impact assessment and results based management
(RBM) plans for Phase-II of CRPs, and provide them an update on SIAC Phase-I activities, including the IEA
evaluation as well as plans for a potential SIAC Phase-II. The discussions at the meeting are expected to feed
into the Phase-II planning. Information about the workshop is available from the SPIA website.
Finally, SPIA is scoping options for a 4-day impact assessment conference to cover all of SIAC outputs and
solicited papers from external researchers, for May or June 2017.
Activity 4.4. Enhancing quality and rigor: Introducing a Star Rating System for IA studies
SPIA launched the online external review and rating system in early April 2016 as a key mechanism for ensuring
high quality assessments of impacts by the CGIAR. The system differs from journal reviews (that focus on
methodological approaches and innovative research) in that it also focuses on criteria such as scale and link to
agricultural research outputs, responding to donor needs. It is not intended to replace journal publications, but
is a systematic way of identifying and showcasing (thereby setting an example of) high-quality work to the CGIAR
and donors. The idea was presented at the Impact Assessment Focal Point (IAFP) meeting in Minneapolis in July
2014 and received with enthusiasm. This document (Criteria for quality rating) outlines the vision for this
system: a significant change in the management process outlined in the draft being that the SPIA Chair will
function as the Chief Editor and supported in that role by SPIA members (as Associate Editors). For each
manuscript submitted for rating, the Chief Editor or one of the Associate Editors will identify external reviewers
and manage the review process with support of SPIA Secretariat staff.
The process started in September 2014 with a contract with the vendor (Allen Press), and a testable version of
the review system online completed in December 2014. The website went live in February 2015. Discussions on
approach to quality rating and criteria continued through 2015, but the process of inviting submissions was
delayed. Since April 2016, SPIA has not received queries about the system or received submissions. The reasons

of 42
for disinterest will be probed at the IAFP meeting in July 2016. SPIA is considering random audits of impact
claims and the associated studies motivating them, as part of a second phase to the SIAC program.
Activity 4.5. CGIAR Impact Website
The CGIAR impact website was re-launched in May 2014. The entire front-end has been redeveloped to enhance
users’ ability to find the information they need, and the visual identity has been brought up to date. New
features such as an impact blog, a latest news section, and a dedicated section for the community of
practitioners of IA, all add useful resources to help increase awareness of IA activity in the CGIAR.
Since then, the Impact website continues to get maintained and updated with information on new calls for
proposals and blog entries. In 2016, a private discussion board was setup to facilitate interactions between
impact assessment scientists at CGIAR Centers/CRPs in relation to capacity building activity 4.2.
A SIAC M & E related activity: Survey of donor demands for impact assessment
In 2014, the Secretariat conducted a ‘Donor Demand for ex post IA’ study as a follow-up to an earlier survey
carried out in 2004-05. This recent survey sought to better understand:
A. How/whether use of epIAs has changed/evolved over time; current perceptions of CGIAR epIAs
B. Are/how some of these changes in use/perceptions relate to actions (e.g. publication of impact briefs)
taken by SPIA on the issues identified in 2005
C. Additional actions required of SPIA, to feed into SPIA strategy. And use this opportunity to solicit
feedback from donors on quality ratings of CGIAR epIAs, methodology and parameters proposed
A brief draft report (Donor demands summary) - qualitative with data summaries, rather than quantitative given
the small number of respondents – was produced. SPIA member J V Meenakshi provided guidance and
assistance in interpreting results of the survey. The report classifies the survey questions into themes and
identifies the key emergent messages. The initial plan was to conduct follow-up interviews via skype/telephone
with individual donors to help clarify ambiguities (similar to the process used for 2004-05 study). However, after
discussing with Rodney Cooke, whom SPIA contacted to lead this component, SPIA decided against a follow-up
at this point given the other priority concerns in the System (SRF revision, second call for CRPs, etc.).
Discussion during the Mid-term Review and SPIA strategy session around the results led to some key messages
emerging and these are highlighted below:
1. Accountability, i.e. demonstrating whether research outputs contribute to development goals, appears
to be the most important purpose for doing epIA. Learning and critical self-analysis are less important
objectives, with the exception of donors like Gates who rate this the highest.
2. While a few donors emphasize rigor and precision in impact estimates, and careful construction of a
counterfactual, many others state that they are not looking for methodological perfection or purity.
Given the long lags between completion of the research and ultimate impacts, they recognize the
difficulty in constructing an appropriate counterfactual, and are of the view that ‘the perfect should not
get in the way of the good’. At the same time, many donors prefer SLO level/large scale impact and
evidence. As one donor stated, this is persuasive communication information. This is consistent with
the donors’ view about the important accountability imperative of epIAs: at scale, such evidence
enables donors to establish that these investments are worthwhile.
3. There is higher demand for epIAs, even with its associated higher costs per study, than for
adoption/uptake studies or for efficacy studies. Science quality/output assessments scores far lower
than the other 3 study types. The Gates allocation was noteworthy in that adoption/uptake studies
received the highest points allocation (60 out of 100 points) followed by efficacy studies (30) and science
quality/output assessments (10). epIAs received 0 points.
4. There is a slight preference for
 epIAs of recent investments, rather than earlier ones
 a suite of epIAs focusing on a small set of indicators, rather than one in-depth study with a rich,
broader set of indicators
 a number of cross-sectional studies (as opposed to a single panel) that may not be in-depth, but
representative of full adoption domain.

of 42
ANNEX 1 - List of SIAC funded studies and timetable (sorted in cronological order by date of the call and grouped, in colour scales, by the SIAC objective)
No. Grantee / Sub-grantee - Partner Donor
SIAC
activity
Project
start
Project
end
Project
extension
Total cost
Paid to
date
Status
% funds
disbursed
Progress
towards
completion
Comment
1 Michigan State University BMGF OBJ.1/2 01.03.13 31.12.15 31.10.16 $2,968,897 $2,968,897 ongoing 100% 85%
2 Michigan State University W1 OBJ.1/2 01.02.15 31.05.16 31.10.16 $649,550 $519,640 ongoing 80% 85%
3 VIRG TECH / CIFOR ISPC OBJ. 4.2 01.01.14 31.12.15 30.11.16 $286,125 $228,022 ongoing 80% 80%
4 ICRISAT / Univ Illinois BMGF OBJ. 4.2 01.10.14 30.09.16 31.12.16 $237,070 $189,656 ongoing 80% 75%
5 CIMMYT ISPC OBJ.3.2 01.01.14 01.03.14 - $19,185 $19,185 completed 100% 100%
6 TUFTS ISPC OBJ.3.2 01.03.14 01.03.16 28.02.16 $49,671 $39,671 completed 80% 100% final payment not made, as per LOA amendment
7 Paris School of Economics ISPC OBJ.3.2 01.01.14 01.03.16 - $76,300 $76,300 completed 100% 100%
8
Innovations for Poverty Action (IPA) / MIT-
JPAL, MIT Sloan
W1 OBJ.3.0 15.07.14 31.12.15 31.12.16 $99,876 $54,932 ongoing 55% 75%
9 ILRI / Emory Univ W1 OBJ.3.0 01.07.14 30.06.16 - $100,000 $30,000 ongoing 30% project on hold until satisfactory progress is made
10 CIMMYT / Univ Goettingen, Univ Geneva W1 OBJ.3.0 01.10.14 30.06.16 - $150,000 $120,000 ongoing 80% 99%
11 CIAT / Virg Tech W1 OBJ.3.0 15.11.14 14.11.16 - $200,000 $160,000 ongoing 80% 80%
12 COLUMBIA / George Washington Univ W1 OBJ.3.0 01.03.15 31.12.16 - $130,119 $104,096 ongoing 80% 75%
13
Qué Funciona para el Desarrollo A.C /
Berkeley, J-PAL, WorldBank
ISPC OBJ.3.2 01.12.14 31.10.16 - $357,608 $286,086 ongoing 80% 85%
14 BERKELEY / Tufts, IRRI BMGF OBJ.3.2 01.05.15 31.12.16 31.05.17 $235,176 $70,553 ongoing 30% 50%
15 YALE / J-PAL, ICIMOD, Dep. of Agric. Nepal BMGF OBJ.3.2 01.05.15 31.12.16 - $268,609 $214,887 ongoing 80% 70%
16 ICARDA / Virg Tech BMGF OBJ.3.1 01.07.15 31.12.16 - $135,468 $108,374 ongoing 80% 70%
17 CIP / Virg Tech ISPC OBJ.3.1 01.06.15 30.11.16 - $200,000 $160,000 ongoing 80% 80%
18 IITA W1/ISPC OBJ.3.1 01.06.15 31.05.17 - $199,978 $159,982 ongoing 80% 55%
19 UC SAN DIEGO / Ram Fishman BMGF OBJ.3.1 01.07.15 31.12.16 - $181,963 $145,571 ongoing 80% 70%
20 IFPRI / WorldBank W1 OBJ.3.1 01.08.15 31.12.16 - $199,967 $159,974 ongoing 80% 70%
21
WORLDFISH / Central Luzon Univ Philippines,
Phiippines Bureau of Fisheries
BMGF OBJ.3.1 01.08.15 31.12.16 - $134,562 $107,650 ongoing 80% 70%
22 TANGO International / IFPRI, Westat BMGF OBJ.3.1 15.10.15 31.08.16 - $199,357 $159,486 ongoing 80% 95%
23 CIAT / Michigan State Univ, Univ Andes W1 /ISPC OBJ.3.3 15.02.16 30.06.17 - $200,000 $60,000 ongoing 30% 30%
24 VIRG TECH / CIFOR ISPC OBJ.3.3 01.03.16 30.06.17 - $140,053 $28,010 ongoing 20% 25%
25 IRRI / North Carolina State Univ / PhilRice W1 OBJ.3.3 15.02.16 30.06.17 - $199,991 $59,997 ongoing 30% 30%
26 ICRAF / Univ Illinois W1 OBJ.3.3 01.02.16 30.06.17 - $199,640 $59,892 ongoing 30% 30%
27 University of Minnesota ISPC OBJ.3.5 15.12.15 15.11.16 - $200,000 $160,000 ongoing 80% 80%
28
Institute for Financial Management and
Research / CIMMYT
BMGF OBJ.2.2 15.05.16 30.06.17 - $199,999 $60,000 ongoing 30% 10%
29
Norwegian University of Life Sciences /
Monica Fisher
BMGF OBJ.2.2 15.05.16 30.06.17 - $111,099 $33,330 ongoing 30% 10%
30 CIMMYT / ICRISAT BMGF OBJ.2.2 15.05.16 30.05.17 - $74,913 $22,474 ongoing 30% 10%
31 Nong Lam University Vietnam / UC Santa Cruz BMGF OBJ.2.2 15.07.16 30.06.17 - $100,970 $0 to begin 0% 0%
32 IFPRI / IITA / GeoPoll BMGF OBJ.2.2 15.07.16 30.06.17 - $199,999 $0 to begin 0% 0%
33 ICRAF / Penn State, Univ Zambia W1 OBJ.2.2 01.06.16 30.06.17 - $190,457 $57,137 ongoing 30% 5%
34 CIMMYT W1 OBJ.2.2 01.06.16 30.06.17 - $74,888 $22,466 ongoing 30% 5%
35 ICRISAT / Univ Illinois / CIMMYT W1 OBJ.2.2 01.07.16 30.06.17 - $149,890 $44,967 ongoing 30% 1%
36 FAO EPIC - informal agreement W1 OBJ.2.2 15.07.16 30.06.17 - $129,400 $0 to begin 0% 0% consultant to be paid in 11 monthly instalments
37 FAO EPIC - informal agreement W1 OBJ.2.2 15.07.16 30.06.17 - $10,000 $0 to begin 0% 0% part of the ICRAF 2.2 LOA

of 42
ANNEX 2
SIAC Objective 2.1. Organize the collection of varietal adoption data – Survey of Participants of the Expert
Elicitation Workshops, July 2016
This survey is being conducted by Michigan State University as the lead on objective 2.1 of the Strengthening
Impact Assessment in CGIAR (SIAC) Project. As part of this objective several Expert Elicitation workshops were
conducted jointly by the CGIAR centers (e.g., CIAT, CIMMYT, CIP, ICRISAT and IRRI) and national partners to
collect varietal adoption data in several crop-country-combinations (CCCs) across South, Southeast and East Asia
in 2014-2016. The purpose of this survey is to assess the approach used to collect crop varietal adoption data
as part of this SIAC activity, and to get some additional information on the status of the seed system for specific
crops. You are asked to respond to this survey because you participated as an expert in one of the Workshops
organized in the past 2 years. You are free to voluntarily choose to participate in this survey, refuse to answer
certain questions, or stop participating at any time without any loss or harm to you. If you choose to participate,
your feedback will help us improved this methodology of collecting adoption data in future. Your responses will
be kept completely confidential to the maximum extent allowable by law. Your responses will be summed
together with those from other workshop participants across more than 100 CCCs. Only general averages from
the analysis will be reported. For any questions about the study, contact Dr. Mywish Maredia from MSU (1-517-
353-6602 or maredia@msu.edu).
By continuing with this survey, you indicate your voluntary consent to participate in this study.
Instructions on submitting the survey: You can complete this Word file, save it on your computer, and email
the file to: maredia@msu.edu.
Alternatively, you can complete this survey online by following this link:
https://form.jotform.com/61745683053156
Your time to complete this survey is very much appreciated.
A. Information about the Expert Elicitation Workshop in which you participated
For which ‘crop-country-combination’ workshop did you participate?
Write your responses in this
column
A1. Crop
A2. Country
A3. If in India or China, specify the State / Province
A4. Year in which this workshop was held
A5. Name of the city where it was held
B. Your background and expertise
B1. Gender |___|
1=Male 2=Female
B2. Highest level of education completed: |____|
1=High school or less 2=Bachelors degree
3=Masters degree 4=Ph.D.
B3. Type of organization you belong (Mark an ‘X’ next
to the response)
1=Government (e.g., Ministry)
2=Public research organization
3=Private sector
4=NGO
5=Independent Farmer
6=Extension
9=Other (specify)
B4. Main area of your expertise (Mark an ‘X’ next to the
response)
1=Plant breeding
2=Other plant sciences (agronomy, pathology, entomology,
etc.)
3=Extension specialist
4=Seed system specialist
5=Socio-economist
6=Marketing
7=Farming
9=Other (specify)

of 42
B5. Number of years associated with this
/organization:
B6. Number of years of experience you have with the
crop mentioned in A1:
B7. Have you ever participated in a similar type of ‘expert elicitation’ process for the purpose of:
a. Estimating varietal adoption? (Yes/no) |___|
b. Estimating the adoption of other types of agricultural technologies? (yes/no) |___|
c. If Yes, please specify the type of technology: ____________________________________
B8. Have you ever participated in a Delphi method of consensus building?
Yes____ / no_____ / don’t know what is Delphi method_____
C. Assessment of the EE workshop
C1. Please share your experience of participating in the above mentioned expert elicitation workshop in terms
of:
a. Strengths
b. Weaknesses
c. Any suggestions you may
have to improve this
methodology
C2. On a scale of 0 to 3 how satisfied were you with the process and methodology of eliciting your expert
opinion during this workshop? (select one option by putting an ‘X’)
___0 =completely dissatisfied ___1=somewhat dissatisfied
___2=somewhat satisfied ___3=completely satisfied.
C3. On a scale of 0 to 3 how confident were you with the adoption estimates at the end of the workshop?
(select one option by putting an ‘X’)
___0 =No confidence at all ___1=somewhat confident
___2=moderately confident ___3=completely confident
C4. In your opinion, how important were the following characteristics/background of participants in the
overall workshop discussion and influencing the results (Scale: 0=Not important and 3=Very important) (Select
your response by putting a ‘X’ for each item a to f listed below)
0=Not
important
1=somewhat
important
2=moderately
important
3=very
important
a. Age
b. Professional position
c. Type or area of expertise
d. Education
e. Religion
f. Gender
C5. Approximately, how much time did you spend on (indicate number of hours):
a. Any preparatory work prior to attending this workshop: ________
b. Time spent traveling to/from your place of residence to the workshop venue: _______
c. Time spent at the workshop: ________
C5. Would you participate in such expert elicitation workshop, if held regularly in future? (Yes/no)
D. Experience with the group discussions
D1. Did the Workshop include forming sub-groups and conducting discussions and estimations at the sub-
group level (yes/no) _________

of 42
D2. How did you come up with the results in the group discussions? (select one option) _______
1=By taking averages of individual results (if this option, respond D2a, and then go the section E)
2=By ‘consensus’ (if this option, respond D2b, and then go section E)
D2a. In case the group estimates did not add up to 100%, how did you proceed? (select one option) ___
1=Adding another minor’ variety to the list
2=Allocating remaining percentages to “other varieties” or local varieties
3=Allocating remaining percentages to major variety
4=Evenly allocating across all varieties
D2b. In case you came up with group estimates by consensus, how was consensus reached? _______
1=Democratic process (more than 50% in favor of a certain estimate/variety)
2=Discussion by convincing the other(s) (by giving examples and evidence)
3=Only certain subgroup members’ opinion counted/ were considered important
E. Assessment of the seed system
E1. How active is the public, private and NGO sector in the following aspects of the seed system for the crop-
country (or state/province) combination noted in A1-A3. Mark ‘X’ in appropriate column
0=Not at all
active
1=Somewhat
active
2=Moderately
active
3=Very active
Public sector
1. Varietal development
2. Seed multiplication
3. Seed dissemination
Private sector
NGO or donor funded programs
E2. In your opinion, what percentage of farmers use harvest from the previous season as seed or planting
material for the production of this crop in this CCC? _____
E3. In your opinion, what percentage of total cropped area is planted every year with new / fresh seed
obtained as ‘seed’ from the market for this CCC? _______
E4. In your opinion how would you rate the overall development of the seed system for this CCC? (select one
option by marking an ‘X’)
___0 =low level of development ___1=somewhat developed
___2=moderately developed ___3=Highly developed
THANK YOU FOR YOUR TIME

SIAC program report

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to SIAC program report (20)

More from Independent Science and Partnership Council of the CGIAR (20)

Recently uploaded (20)

SIAC program report