SlideShare a Scribd company logo
Developmental Evaluation: Tips, Tools,
& Lessons Learned
Diálogo Improbable
30 May 2019
Joshua Fisher
Our conversation
• What is Developmental Evaluation?
• Why do we need it?
• What are useful tools?
• What are good practices?
Our conversation
What is Developmental Evaluation?
What is Developmental Evaluation?
Analytical system of information collection, sense-making, evidence-
based decision support
1. Process monitoring
2. Outcome evaluation
3. Adaptive management
(Davies & Dart 2005)
(Gamble 2008)
(Reproduced from Gamble 2008)
Our conversation
Why do we need it?
What types of problems do we encounter in development and social innovation?
Complex:
Cause & Effect: Understandable
in retrospect, but not repeatable
Complicated:
Cause & Effect: Detectable but not
separate over time and space
Chaotic:
Cause & Effect: Not identifiable
Simple:
Cause & Effect: Repeatable, perceivable
and predictable
(Snowden & Boone 2005)
Characteristics
• Boundaries and limits of problems are clear
• Problems are well-understood by stakeholders
• Problem & Solution behave in predictable ways
• Finite set of solutions
(Reproduced from Gamble 2008)
http://www.migration4development.org/docs/logframe.pdf
Complex:
Cause & Effect: Understandable
in retrospect, but not repeatable
Complicated:
Cause & Effect: Detectable but not
separate over time and space
Chaotic:
Cause & Effect: Not identifiable
Simple:
Cause & Effect: Repeatable, perceivable
and predictable
(Snowden & Boone 2005)
Characteristics
• Boundaries and limits of problems are unclear
• Problems defined uniquely by each stakeholder
• Dynamics & interactions are unpredictable
• Infinite solution set
Development challenges demonstrate BOTH complicated & complex dynamics
(Reproduced from Gamble 2008)(Reproduced from Gamble 2008)
Today’s development challenges occur in complex systems
Interconnectivity
Non-linear cause
& effect
Multi-causality
Constant change
Complex Systems are multi-scalar
Complexity is replicated at each scale
Interactions across scales
Solutions have to address this fractal pattern
(Fisher & Coleman Forthcoming)
Not solved, but re-solved over & over
Poorly defined
Socially complex
Evolving
Instigating
Non-ending
(Balint et al 2011)
Our ‘development challenges’ are actually ‘wicked problems’
(Balint et al 2011 – Wicked Environmental Problems)
Collectively define the problem
Discover drivers & dynamics
Discuss & implement
Learn:
• What happens
• What’s next
• What else
• Who else
• Why
’Re-solving’ wicked problems requires deliberative & adaptive intervention
Adapt & iterate
Wicked problems require changing the way we think about
relationships
Individual participant/partner in programming
Entrepreneurs, Market linkages, etc.
Network of convening & coordinating partners
Platform members, design team
Partners in the operating ecosystem
Governments, families, social networks, sectors
Convening partners
Investors, capital providers
Beneficiary
Grantee / Implementing body
Secondary / Indirect beneficiaries
For donor, sponsor
’Partnerships’ require that everyone contribute to
monitoring, evaluation, and learning
The ‘Blindspots’ of Traditional M&E/Impact Assessment (USAID, 2013)
1) Broad range of action & outcomes in system (non-linear, cross-scale, unintended)
2) Alternative causes and influence from other actors & factors (attribution dilemma)
3) Full-range of processes & pathways that contribute to observed change (multi-causality)
4) Some outcomes are unobservable (non-occurrence, inference)
http://www.migration4development.org/docs/logframe.pdf
Performance-oriented Methods
Logical frameworks (equivalent)
• Objectives
• Measured against preset targets
• Quarterly data collection and ex-
post data assessment and
reporting to sponsor
Process-oriented Methods
Observe & reflect on processes
• Allow for emergent
outcomes
• Continuous data collection;
periodic assessment &
discussion with partners
• Situate results analysis &
evaluation in system context
Deliberative Learning Ecosystem
Our conversation
What are useful tools?
GOPC 2014 – Spaces MERL
Long-term development investment to encourage entrepreneurship &
job creation in India, Mozambique, & Peru
Platform approach: partnership with local communities and
implementing partners
Phases of intervention
1. Listening
2. Co-creation
3. Prototyping
4. Acceleration & Deepening Impact
5. Continued Support
How can each platform understand their own ‘wicked’ system?
How can we compare across different cases?
Developmental Evaluation Approach
Evaluation Question
1) Does participation in an innovation-based development initiative lead to direct (tangible goods, asset access and ownership, income, etc.) and/or
indirect economic benefits (access to services, employability, additive market linkages, etc.) for participating households, communities, and enterprises?
2) Are the prototype experiments developed through the co-creation process in each W4P intervention qualitatively and/or technically similar, and do
they address the issues raised during the ‘listening’ phase?
3) What strategies are employed by innovation networks/platforms to adapt to structural and contextual changes?
4) How does participation in an innovation network/platform affect interrelationships among implementing partners, and what are the costs and
benefits of this approach when compared with more traditional development projects?
5) How does the presence of an innovation-based initiative affect the wider communities and members of the development ecosystem?
6) What additional design considerations are required to more effectively monitor, evaluate, measure and learn from innovation platforms?
Dashboards
(Mayo 2016)
Needs /
Opportunities
Matrix
Most Significant
Change
Barriers /
Enablers Matrix
Sentinel
Indicators
Developmental Evaluation Approach
Developmental Evaluation: Tips, Tools,  & Lessons Learned.
Overview of Tools
Most Significant Change
(Davies & Dart 2005)
Most Significant Change
Prototype
participant
Prototype
participant
Prototype
participant
Prototype
participant
Prototype
participant
Prototype
participant
Prototype
participant
Prototype
participant
Prototype
participant
W4P Support
Team
W4P Support
Team
W4P Support
Team
W4P Support
Team
W4P Support
Team
W4P Support
Team
W4P
Platform
Member /
Organization
W4P
Platform
Member /
Organization
W4P
Platform
Member /
Organization
W4P Country Platform
W4P Regional Platform W4P Regional Platform
W4P Program (LCBF, 3 Country Teams, Other Partners)
SignificantChangeNarratives
Feedbacksentbackthroughdecisionchain
• Significant
Change
narratives
collected
• Support teams
select narratives
to send up
• Narratives discussed and
responses and lessons
learned documented
• Platform member
organizations select
narratives to send up
• Narratives discussed and
responses and lessons
learned documented
• Country platform selects
narratives to send up
• Country teams and LCBF
select narratives to share
with other platforms
Developmental Evaluation: Tips, Tools,  & Lessons Learned.
Developmental Evaluation: Tips, Tools,  & Lessons Learned.
May / June ‘19 Oct / Nov ‘19 Feb / Mar ‘20
2019 2020 2021 2022
Develop a deeper understanding over time & track changes
Overview of Tools
Sentinel Indicators
Source:
(Mayo 2016)
Overview of Tools
Examples
Source:
(Mayo 2016)
Our conversation
What are good practices?
Multiple methods, multiple scales, with intentional discussions to inform programming
Evaluation Partners need investment/ownership & distance/objectivity
Cost-efficient in terms of ALL partners’ time, energy, economies, incentives, ROI & SROI
Balance between scientific purity/rigor & useful data and information
Partnerships in learning & adaptation rather than transactional programs & evaluations
Good Practices
References
• Balint, P.J., R.E. Stewart, A. Desai, and L.C. Walters. 2011. Wicked Environmental Problems: Managing Uncertainty and Conflict. Washington, D.C.: Island Press.
• Davies, R., and J. Dart. (2005) The Most Significant Change (MSC) Technique. Available online at:
https://www.researchgate.net/publication/275409002_The_'Most_Significant_Change'_MSC_Technique_A_Guide_to_Its_Use/download
• Fisher, J., & Coleman, P. (Forthcoming). The fractal nature of intractable conflict: Implications for sustainable transformation. In L. Kriesberg (Ed.) Transforming
Intractable Conflicts. Rowman and Littlefield. Boulder.
• Gamble, J. (2008). A Developmental Evaluation Primer. Montreal. The J.W. McConnell Family Foundation.
• GOPC (2016) Spaces MERL – Systems and Complexity White Paper. Washington DC. USAID.
• Mayo, T. (2016) AEA 2015 - Sentinel Indicators: A Systems-Based Approach to Monitoring and Evaluation. USAID.
• Snowden, D. J. & M.E. Boone (2007). A Leaders Framework for Decision Making, Harvard Business Review
• USAID, 2013. Discussion Note: Complexity Aware Monitoring. V 2.0. Washington DC. Available online at:
http://www.dmeforpeace.org/sites/default/files/0107_Complexity%20Aware%20Monitoring%202013-12-11%20FINAL.pdf
• USAID, 2018. Discussion Note: Complexity Aware Monitoring. Washington DC. Available online at: http://usaidlearninglab.org/library/complexity-aware-
monitoring-discussion-note-brief
• http://www.migration4development.org/docs/logframe.pdf
Discussion

More Related Content

PPTX
Monitoring & Evaluation - the RAPID approach
PPT
Presentation
PPT
Outcome Mapping for Insight to Impact Meeting
PPTX
Dr. Mark Reed What is participation
PPTX
Randomized Controlled Trials: Insights for Civic Tech
PDF
Van Ael, Monastiridis, Tarquini, Nogueira Systemic Design Toolkit Workshop
PDF
Balancing Acceleration and Systemic Impact
PPT
06 Om At Training W Kshop Org Practices & Monitoring
Monitoring & Evaluation - the RAPID approach
Presentation
Outcome Mapping for Insight to Impact Meeting
Dr. Mark Reed What is participation
Randomized Controlled Trials: Insights for Civic Tech
Van Ael, Monastiridis, Tarquini, Nogueira Systemic Design Toolkit Workshop
Balancing Acceleration and Systemic Impact
06 Om At Training W Kshop Org Practices & Monitoring

What's hot (20)

PDF
Evaluating Problem Gambling KTE
PDF
Requirements Engineering for Sustainability - Open Lecture at LUT 2016
PPTX
Vanessa wilkes tsrc
PDF
Key Trends in Alternative Approaches to Monitoring and Evaluating SBC SCHOOLEY
PDF
Amos OPPI Workshop October 12 2011 Final
PPTX
A new direction for infrastructure (NCVO Annual Conference 2012)
PDF
Stakeholder management in a matrix organisation - 25th August 2015
PPTX
CBODN 2013 unleashing results in groups tackling wicked problems
PPT
ACES Participatory Methods Training 2011
PDF
Coalition Assessment: Approaches for Measuring Capacity and Impact
PPT
Review of Communications in DFID-funded Research Programme Consortia (RPCs)
PPTX
Managing a Project with Your Team - Lou Bergner
PDF
Alternative approaches, follow the feedback, plan for adaptation rahaim updated
PPTX
A reflection on connecting complexity theory and design for policy
PDF
Pioneering-New-Operating-Models-and-Measurement-Techniques-for-Private-Sector...
PDF
Team Collaboration Competencies for Complex Social Challenges
PDF
Design by Doing in Louisiana Farmers Markets
PPTX
How the Architecture decision methods deal with Group Decision Making
PDF
Shared Rationales in Group Activities
Evaluating Problem Gambling KTE
Requirements Engineering for Sustainability - Open Lecture at LUT 2016
Vanessa wilkes tsrc
Key Trends in Alternative Approaches to Monitoring and Evaluating SBC SCHOOLEY
Amos OPPI Workshop October 12 2011 Final
A new direction for infrastructure (NCVO Annual Conference 2012)
Stakeholder management in a matrix organisation - 25th August 2015
CBODN 2013 unleashing results in groups tackling wicked problems
ACES Participatory Methods Training 2011
Coalition Assessment: Approaches for Measuring Capacity and Impact
Review of Communications in DFID-funded Research Programme Consortia (RPCs)
Managing a Project with Your Team - Lou Bergner
Alternative approaches, follow the feedback, plan for adaptation rahaim updated
A reflection on connecting complexity theory and design for policy
Pioneering-New-Operating-Models-and-Measurement-Techniques-for-Private-Sector...
Team Collaboration Competencies for Complex Social Challenges
Design by Doing in Louisiana Farmers Markets
How the Architecture decision methods deal with Group Decision Making
Shared Rationales in Group Activities
Ad

Similar to Developmental Evaluation: Tips, Tools, & Lessons Learned. (20)

PPTX
Reflexive monitoring in action
PPT
Workshop: Monitoring, evaluation and impact assessment
PDF
Evaluation approaches presented by hari bhusal
PDF
How to evaluate in complex and adverse settings
KEY
A Case Study on the Use of Developmental Evaluation for Navigating Uncertain...
PDF
Developmental Evaluation for Social Innovation
PPTX
FCAS M&E Seminar
PPTX
Using empowerment evaluation to strengthen talent search progamming march 2011
PPTX
concept of evaluation "Character insight"
PDF
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
PPTX
Design for complexity, using evaluative methods
PPTX
Kate McKegg and Nan Wehipeihana (2010). A practitioners introduction to Devel...
PPTX
Evaluability Assessments and Choice of Evaluation Methods
PDF
Insights on Using Developmental Evaluation for Innovating: A Case Study on th...
PPT
Unit 3- Theories and approaches in Monitoring and Evaluation.ppt
PPTX
German Fundraising Congress 2015 - Disruptive Change Workshop
PDF
Evaluation to Improve Development Results
PDF
Response to the Day and Ways Forward SARRIOT
PDF
IDS Impact Innovation and Learning Workshop March 2013: Day 1, Paper Session ...
PPTX
PPWNov13- Day 2 keynote- S.Sridharan- U Toronto
Reflexive monitoring in action
Workshop: Monitoring, evaluation and impact assessment
Evaluation approaches presented by hari bhusal
How to evaluate in complex and adverse settings
A Case Study on the Use of Developmental Evaluation for Navigating Uncertain...
Developmental Evaluation for Social Innovation
FCAS M&E Seminar
Using empowerment evaluation to strengthen talent search progamming march 2011
concept of evaluation "Character insight"
IDS Impact, Innovation and Learning Workshop March 2013: Day 2, Keynote 2 Pat...
Design for complexity, using evaluative methods
Kate McKegg and Nan Wehipeihana (2010). A practitioners introduction to Devel...
Evaluability Assessments and Choice of Evaluation Methods
Insights on Using Developmental Evaluation for Innovating: A Case Study on th...
Unit 3- Theories and approaches in Monitoring and Evaluation.ppt
German Fundraising Congress 2015 - Disruptive Change Workshop
Evaluation to Improve Development Results
Response to the Day and Ways Forward SARRIOT
IDS Impact Innovation and Learning Workshop March 2013: Day 1, Paper Session ...
PPWNov13- Day 2 keynote- S.Sridharan- U Toronto
Ad

More from Innovation and Technology for Development Centre (20)

PDF
INFRAESTRUCTURAS VERDES CAMBIO CLIMÁTICO DESARROLLO SOSTENIBLE
PDF
Presentación Giulio Quaggiotto-Diálogo improbable .pptx.pdf
PPTX
Presentación Antonio Zarate- Jornada.pptx
PPTX
Presentación Carlos Gregorio Jornada .pptx
PPTX
Presentación Julián Briz. Jornada 23 mayo.pptx
PPTX
Enfoques de portafolio para la innovación: Navegar por la complejidad de los ...
PPTX
Dialogo (im)probable. ¿Cómo aumentar la aportación de electricidad renovable ...
PPTX
La dimensión democrática de la transición justa
PDF
Colaboración para la transición climática urbana en Ucrania
PPTX
Filantropía comunitaria para fortalecer vínculos desde lo local
PDF
PDF
Entornos escolares como motores de cambio hacia la acción climática
PPTX
Hacia un sistema de movilidad urbana sostenible
PDF
PDF
Infraestructuras verdes urbanas en España. Realidades y percepciones. Agricul...
PDF
Timber Demand Study Report. Structures and façade - HCC EU CINCO
PDF
Bio-based Construction. An illustrative economic and environmental impact ana...
PDF
EDIFICIOS CON MATERIALES BIOLÓGICOS Y DISEÑO CIRCULAR PARA CIUDADES SALUDABLE...
INFRAESTRUCTURAS VERDES CAMBIO CLIMÁTICO DESARROLLO SOSTENIBLE
Presentación Giulio Quaggiotto-Diálogo improbable .pptx.pdf
Presentación Antonio Zarate- Jornada.pptx
Presentación Carlos Gregorio Jornada .pptx
Presentación Julián Briz. Jornada 23 mayo.pptx
Enfoques de portafolio para la innovación: Navegar por la complejidad de los ...
Dialogo (im)probable. ¿Cómo aumentar la aportación de electricidad renovable ...
La dimensión democrática de la transición justa
Colaboración para la transición climática urbana en Ucrania
Filantropía comunitaria para fortalecer vínculos desde lo local
Entornos escolares como motores de cambio hacia la acción climática
Hacia un sistema de movilidad urbana sostenible
Infraestructuras verdes urbanas en España. Realidades y percepciones. Agricul...
Timber Demand Study Report. Structures and façade - HCC EU CINCO
Bio-based Construction. An illustrative economic and environmental impact ana...
EDIFICIOS CON MATERIALES BIOLÓGICOS Y DISEÑO CIRCULAR PARA CIUDADES SALUDABLE...

Recently uploaded (20)

PPTX
Computer network topology notes for revision
PDF
Introduction to Data Science and Data Analysis
PPTX
Introduction to Knowledge Engineering Part 1
PPTX
SAP 2 completion done . PRESENTATION.pptx
PDF
Business Analytics and business intelligence.pdf
PPT
ISS -ESG Data flows What is ESG and HowHow
PPTX
IB Computer Science - Internal Assessment.pptx
PPTX
STERILIZATION AND DISINFECTION-1.ppthhhbx
PDF
annual-report-2024-2025 original latest.
PDF
[EN] Industrial Machine Downtime Prediction
PPTX
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
PPTX
Market Analysis -202507- Wind-Solar+Hybrid+Street+Lights+for+the+North+Amer...
PPTX
iec ppt-1 pptx icmr ppt on rehabilitation.pptx
PPTX
01_intro xxxxxxxxxxfffffffffffaaaaaaaaaaafg
PPTX
IBA_Chapter_11_Slides_Final_Accessible.pptx
PPTX
Database Infoormation System (DBIS).pptx
PPT
Reliability_Chapter_ presentation 1221.5784
PPTX
AI Strategy room jwfjksfksfjsjsjsjsjfsjfsj
PPTX
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
PDF
Galatica Smart Energy Infrastructure Startup Pitch Deck
Computer network topology notes for revision
Introduction to Data Science and Data Analysis
Introduction to Knowledge Engineering Part 1
SAP 2 completion done . PRESENTATION.pptx
Business Analytics and business intelligence.pdf
ISS -ESG Data flows What is ESG and HowHow
IB Computer Science - Internal Assessment.pptx
STERILIZATION AND DISINFECTION-1.ppthhhbx
annual-report-2024-2025 original latest.
[EN] Industrial Machine Downtime Prediction
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
Market Analysis -202507- Wind-Solar+Hybrid+Street+Lights+for+the+North+Amer...
iec ppt-1 pptx icmr ppt on rehabilitation.pptx
01_intro xxxxxxxxxxfffffffffffaaaaaaaaaaafg
IBA_Chapter_11_Slides_Final_Accessible.pptx
Database Infoormation System (DBIS).pptx
Reliability_Chapter_ presentation 1221.5784
AI Strategy room jwfjksfksfjsjsjsjsjfsjfsj
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
Galatica Smart Energy Infrastructure Startup Pitch Deck

Developmental Evaluation: Tips, Tools, & Lessons Learned.

  • 1. Developmental Evaluation: Tips, Tools, & Lessons Learned Diálogo Improbable 30 May 2019 Joshua Fisher
  • 2. Our conversation • What is Developmental Evaluation? • Why do we need it? • What are useful tools? • What are good practices?
  • 3. Our conversation What is Developmental Evaluation?
  • 4. What is Developmental Evaluation? Analytical system of information collection, sense-making, evidence- based decision support 1. Process monitoring 2. Outcome evaluation 3. Adaptive management (Davies & Dart 2005) (Gamble 2008)
  • 7. What types of problems do we encounter in development and social innovation? Complex: Cause & Effect: Understandable in retrospect, but not repeatable Complicated: Cause & Effect: Detectable but not separate over time and space Chaotic: Cause & Effect: Not identifiable Simple: Cause & Effect: Repeatable, perceivable and predictable (Snowden & Boone 2005) Characteristics • Boundaries and limits of problems are clear • Problems are well-understood by stakeholders • Problem & Solution behave in predictable ways • Finite set of solutions
  • 8. (Reproduced from Gamble 2008) http://www.migration4development.org/docs/logframe.pdf
  • 9. Complex: Cause & Effect: Understandable in retrospect, but not repeatable Complicated: Cause & Effect: Detectable but not separate over time and space Chaotic: Cause & Effect: Not identifiable Simple: Cause & Effect: Repeatable, perceivable and predictable (Snowden & Boone 2005) Characteristics • Boundaries and limits of problems are unclear • Problems defined uniquely by each stakeholder • Dynamics & interactions are unpredictable • Infinite solution set
  • 10. Development challenges demonstrate BOTH complicated & complex dynamics (Reproduced from Gamble 2008)(Reproduced from Gamble 2008)
  • 11. Today’s development challenges occur in complex systems Interconnectivity Non-linear cause & effect Multi-causality Constant change
  • 12. Complex Systems are multi-scalar Complexity is replicated at each scale Interactions across scales Solutions have to address this fractal pattern (Fisher & Coleman Forthcoming)
  • 13. Not solved, but re-solved over & over Poorly defined Socially complex Evolving Instigating Non-ending (Balint et al 2011) Our ‘development challenges’ are actually ‘wicked problems’
  • 14. (Balint et al 2011 – Wicked Environmental Problems) Collectively define the problem Discover drivers & dynamics Discuss & implement Learn: • What happens • What’s next • What else • Who else • Why ’Re-solving’ wicked problems requires deliberative & adaptive intervention Adapt & iterate
  • 15. Wicked problems require changing the way we think about relationships Individual participant/partner in programming Entrepreneurs, Market linkages, etc. Network of convening & coordinating partners Platform members, design team Partners in the operating ecosystem Governments, families, social networks, sectors Convening partners Investors, capital providers Beneficiary Grantee / Implementing body Secondary / Indirect beneficiaries For donor, sponsor ’Partnerships’ require that everyone contribute to monitoring, evaluation, and learning
  • 16. The ‘Blindspots’ of Traditional M&E/Impact Assessment (USAID, 2013) 1) Broad range of action & outcomes in system (non-linear, cross-scale, unintended) 2) Alternative causes and influence from other actors & factors (attribution dilemma) 3) Full-range of processes & pathways that contribute to observed change (multi-causality) 4) Some outcomes are unobservable (non-occurrence, inference) http://www.migration4development.org/docs/logframe.pdf
  • 17. Performance-oriented Methods Logical frameworks (equivalent) • Objectives • Measured against preset targets • Quarterly data collection and ex- post data assessment and reporting to sponsor Process-oriented Methods Observe & reflect on processes • Allow for emergent outcomes • Continuous data collection; periodic assessment & discussion with partners • Situate results analysis & evaluation in system context Deliberative Learning Ecosystem
  • 18. Our conversation What are useful tools?
  • 19. GOPC 2014 – Spaces MERL
  • 20. Long-term development investment to encourage entrepreneurship & job creation in India, Mozambique, & Peru Platform approach: partnership with local communities and implementing partners Phases of intervention 1. Listening 2. Co-creation 3. Prototyping 4. Acceleration & Deepening Impact 5. Continued Support
  • 21. How can each platform understand their own ‘wicked’ system? How can we compare across different cases?
  • 22. Developmental Evaluation Approach Evaluation Question 1) Does participation in an innovation-based development initiative lead to direct (tangible goods, asset access and ownership, income, etc.) and/or indirect economic benefits (access to services, employability, additive market linkages, etc.) for participating households, communities, and enterprises? 2) Are the prototype experiments developed through the co-creation process in each W4P intervention qualitatively and/or technically similar, and do they address the issues raised during the ‘listening’ phase? 3) What strategies are employed by innovation networks/platforms to adapt to structural and contextual changes? 4) How does participation in an innovation network/platform affect interrelationships among implementing partners, and what are the costs and benefits of this approach when compared with more traditional development projects? 5) How does the presence of an innovation-based initiative affect the wider communities and members of the development ecosystem? 6) What additional design considerations are required to more effectively monitor, evaluate, measure and learn from innovation platforms?
  • 23. Dashboards (Mayo 2016) Needs / Opportunities Matrix Most Significant Change Barriers / Enablers Matrix Sentinel Indicators Developmental Evaluation Approach
  • 25. Overview of Tools Most Significant Change (Davies & Dart 2005)
  • 26. Most Significant Change Prototype participant Prototype participant Prototype participant Prototype participant Prototype participant Prototype participant Prototype participant Prototype participant Prototype participant W4P Support Team W4P Support Team W4P Support Team W4P Support Team W4P Support Team W4P Support Team W4P Platform Member / Organization W4P Platform Member / Organization W4P Platform Member / Organization W4P Country Platform W4P Regional Platform W4P Regional Platform W4P Program (LCBF, 3 Country Teams, Other Partners) SignificantChangeNarratives Feedbacksentbackthroughdecisionchain • Significant Change narratives collected • Support teams select narratives to send up • Narratives discussed and responses and lessons learned documented • Platform member organizations select narratives to send up • Narratives discussed and responses and lessons learned documented • Country platform selects narratives to send up • Country teams and LCBF select narratives to share with other platforms
  • 29. May / June ‘19 Oct / Nov ‘19 Feb / Mar ‘20 2019 2020 2021 2022 Develop a deeper understanding over time & track changes
  • 30. Overview of Tools Sentinel Indicators Source: (Mayo 2016)
  • 32. Our conversation What are good practices?
  • 33. Multiple methods, multiple scales, with intentional discussions to inform programming Evaluation Partners need investment/ownership & distance/objectivity Cost-efficient in terms of ALL partners’ time, energy, economies, incentives, ROI & SROI Balance between scientific purity/rigor & useful data and information Partnerships in learning & adaptation rather than transactional programs & evaluations Good Practices
  • 34. References • Balint, P.J., R.E. Stewart, A. Desai, and L.C. Walters. 2011. Wicked Environmental Problems: Managing Uncertainty and Conflict. Washington, D.C.: Island Press. • Davies, R., and J. Dart. (2005) The Most Significant Change (MSC) Technique. Available online at: https://www.researchgate.net/publication/275409002_The_'Most_Significant_Change'_MSC_Technique_A_Guide_to_Its_Use/download • Fisher, J., & Coleman, P. (Forthcoming). The fractal nature of intractable conflict: Implications for sustainable transformation. In L. Kriesberg (Ed.) Transforming Intractable Conflicts. Rowman and Littlefield. Boulder. • Gamble, J. (2008). A Developmental Evaluation Primer. Montreal. The J.W. McConnell Family Foundation. • GOPC (2016) Spaces MERL – Systems and Complexity White Paper. Washington DC. USAID. • Mayo, T. (2016) AEA 2015 - Sentinel Indicators: A Systems-Based Approach to Monitoring and Evaluation. USAID. • Snowden, D. J. & M.E. Boone (2007). A Leaders Framework for Decision Making, Harvard Business Review • USAID, 2013. Discussion Note: Complexity Aware Monitoring. V 2.0. Washington DC. Available online at: http://www.dmeforpeace.org/sites/default/files/0107_Complexity%20Aware%20Monitoring%202013-12-11%20FINAL.pdf • USAID, 2018. Discussion Note: Complexity Aware Monitoring. Washington DC. Available online at: http://usaidlearninglab.org/library/complexity-aware- monitoring-discussion-note-brief • http://www.migration4development.org/docs/logframe.pdf