IRH Lessons Learned in "Embracing Scale Up Error" in Five Countries
1 like285 views
This document summarizes lessons learned from scaling up the Standard Days Method family planning intervention in 5 countries over 10 years. Key lessons include: (1) Define scale-up outcomes operationally and contextually; (2) Tools and guidance may need further simplification for scale-up; (3) Repeat evidence of the method's effectiveness throughout; (4) Start promotion early and ensure parity with other methods; (5) Develop temporary monitoring until integrated into health systems; and (6) Continually monitor that gains are not reversed by changing systems. Documenting scale-up challenges can help refine knowledge and help others.
IRH Lessons Learned in "Embracing Scale Up Error" in Five Countries
- 1. IRH Lessons Learned in ‘Embracing Scale Up Error’ in Five Countries Marie Mukabatsinda, Susan Igras, Arsene Binanga, Rebecka Lundgren, Victoria Jennings Institute for Reproductive Health Georgetown University International FP Conference November 30, 2011 Dakar, Senegal
- 2. Standard Days Method: From Research to Practice Scale-Up Integration Case Studies Studies 2007-2013 Operations 2005 - 2007 Research 2003- 2005 Pilot Studies 2000-2004 Method Concept & Efficacy Trial 1998-2002
- 3. 5 years of systematic scale up and accompanying research in 5 countries • Expanding access to SDM at national / regional scale in FP Systems integration programs • DR Congo, Guatemala, India, Mali, and Rwanda Political support & • Even with strong pilots technical leadership Services Integration and scale up planning, during different scale up phases, different things can go wrong!
- 4. Scaling up intervention and research informed by conceptual framework • ExpandNet/WHO scaling up framework • Scale up processes and outcomes measured The Innovation Scaling-up User The through Resource Team Strategy Organization(s) elements of scaling up – pre/post scale up research at household and facility level, TYPE OF SCALING UP – FP stakeholder interviews – benchmark tracking, Strategic choice – key events monitoring, MONITORING areas DISSEMINATION ORGANIZATIONAL COSTS/RESOURCE AND AND ADVOCACY PROCESS MOBILIZATION EVALUATION – Most Significant Change story collection
- 5. Overlapping phases, not discrete steps… Source: PATH’s product development process
- 7. What is scale up success? In addition to integration into norms and sub-systems… SDM scale up goals by country Guatemala SDM offered in at least 1/6 of the country (3 departments) DRC SDM offered in at least 50% of country (ie, in 250 of 515 health zones, accompanying GoDRC efforts to re-establish national FP program) India SDM/LAM availability at facility and community level in 11/24 districts in Jharkhand State 40% of WRA in these districts have heard of SDM and LAM Mali Reinforce SDM integration in 90% of public, private, and community SDPs in 8 regions of Mali and the district of Bamako. Rwanda SDM integrated into at least 95% of SDPs and at least 20% of pharmacies and private clinics supported by PSI. (1) Define SU outcomes operationally and within context!
- 8. Ensuring fidelity of core elements of SDM remains intact once it begins to go to scale • SDM scale up ‘package’ includes product (CycleBeads), training and supervision tools, IEC materials, etc. • During pilot phase in Peru, Rwanda, and India, research indicated that SDM could be easily offered and used. • Early scale up phase used pilot phase materials/tools, but lower level providers had problems understanding training, AND, • Training too long to be integrated into national FP training curricula. (2) You might think • Response: Simplified, shortened training and provider and validated curricula and tools tools are simple to facilitate integration. enough, but will need to simplify even more for scale up
- 9. Each innovation faces unique social (3) Repeat method and political challenges, even if evidence throughout intro defined as a ‘priority intervention’ by and expansion the MOH phases • SDM can challenge core beliefs of some providers and managers about ‘what is modern contraception,’ which can be managed during pilot stage via one-on-one discussions • Going to scale, individual discussions were impossible, which led to re-emergence of skepticism about SDM and its efficacy (confusion with other NFPs) • Response: District ‘SDM focal points’ strategy plus renewed advocacy
- 10. Competition with Other Priority FP Interventions • New methods need to develop a ‘social reputation’ to succeed • Focused on systems strengthening versus awareness creation in early years. • With limited resources, SDM promotion cannot compete with other underused methods with more resources. • Question: Finding the supply- (4) Start early and demand balance – When? How match SDM promotion much? efforts to level of others to create equity.
- 11. Until New Best Practices are Institutionalized… • Pilot studies showed important results • Going to scale, need evidence that SDM investment is worth the system investment (measured most often as new FP users) (5) Until integrated into HMIS, develop • BUT - HMIS system revisions occur only temporary user & QA periodically, government info is unavailable, monitoring systems to and need to reassure policy makers of their have timely evidence scale up investment of return on investment • AND – quality questions will often arise • Response: Develop a comprehensive quality assurance approach to collect service data and quality of service delivery and end use data
- 12. Scale up monitoring cannot use a simple checklist approach • Opportunities (‘seize the moment’) can lead to quick gains in institutionalization. STILL, never assume once something is institutionalized it never changes. • Rwanda (6) Given dynamic nature of systems, – Between 2009 and 2010, inclusion of need to monitor gains SDM/LAM in two important systems to ensure they are not (finance and HMIS) were reversed, reversed – Which led to dis-incentives & -legitimization (if not counted, not important). • Response: Advocacy. Still, cyclical nature of systems revisions means it takes time to reverse reporting rules
- 13. (1) Define SU outcomes (6) Monitor systems operationally and within gains to ensure they context! (4) Start early and are not reversed match SDM (2) You will need to promotion efforts to simplify tools and level of others to guidance even create equity. more for scale up - - - - - - - - - - AND THROUGHOUT - - - - - - - - - (3) Repeat method evidence throughout intro and expansion phases (5) Until HMIS-integrated, temporary user & QA monitoring systems give evidence of results
- 14. Final thoughts • Need to recognize – Constantly changing systems and political environments lead to implementation challenges, even with good scale up planning. – As a multi-year and multi-organization process, you cannot control all scale up efforts or consequences. • Those involved in systematic scale up should document implementation surprises, miscalculations, and incorrect assumptions at different phases of scale up • To contribute to refining knowledge of common scale up threats and help others to avoid them in the future.
Editor's Notes
- #3: After over a decade of testing the Standard Days Method, we are now working on scaling up the method with partners into FP programs.
- #6: One nice way to think about taking a new FP method to scale is a schematic developed by PATH to new product diffusion.You’ll see that scale up is a series of waves.The ‘innovate’ wave is the conceptualization and testing of the new FP innovation.‘Introduction’ wave is the first expansion phase.If all goes well, full scale ‘integration’ phase begins.Throughout all waves it important to advocate for inclusion into programs, collaborate with partners (since no one organization can take an innovation to scale by itself), and to communicate results as they are gathered (since policy makers and program managers and service providers need to know that the scale up effort is leading to results and is worthwhile to take to scale.
- #7: But in reality, scaling up implementation can be more like a game of chutes and ladders. There are opportunities that arise can move scale up more quickly (ladders). There are also unexpected events that may take you backward a step or two. But with a clear scale up goal in mind, these should be considered just part of the scale up process (the ‘finish’ line in the top left of the game board.For this presentation, I am going to share with you some of the miscalculations and incorrect assumptions that we made at different phases of scale up, some of the surprises we experienced, and what we did to adjust efforts so scale up could continue forward.
- #8: As SU begins, it’s important to define your scale up outcomes and operationalize them in the context of the country and health care system. We knew that SU involved institutionalization into norms and standards, but in terms of services availability, SU outcomes needed to be stated in precise ways – for measurement purposes as well as political purposes.You see the different scale up goals by country. All countries define SU goals differently and it is very important to listen to stakeholders and ensure that scale up is worded in ways that make sense.In DRC, for example, as the government is re-establishing its health care and FP programs, we linked SDM SU goals to government FP goals in terms of expansion.Is scale up purely clinic based or also community based (India example)?Can you talk about scale up as scale up, or does it need to be described as Integration. (Mali example. The MOH expressed that SU was completed since the SDM was in norms and standards. What was occurring now was integration into services.)Are we working only to scale up in the public sector, or also to reach private sector? (Rwanda example)
- #9: While we say we are scaling up SDM, you need to think of SU more as a package of activities/materials used for scale up of the product/service.One of our early miscalculations was related to training, a core component of scale up efforts. We thought that materials used during the pilot could be used during SU – our research showed us that providers could offer the method easily and that it was simple for users to learn - but that was not the case.ALSO, we had developed great curriculum for training providers involved in the introduction studies, but there curricula were too long and not possible to integrate into FP CTU curricula. RESPONSE: Validation needed to make sure the simplified training still led to quality service offering.LESSON LEARNED.
- #10: You cannot assume that if the central MOH, based on research and evidence, decides it wants to add SDM, that this means there will not be opposition or scepticism about the new method.SDM is a natural and modern method, and this challenges some people! Skepticism can and was managed when dealing with a small set of people in the pilot years. As expansion began, though, it was not possible to maintain one-on-one advocacy. We assumed that one or two presentation on the scientific evidence and efficacy of the method would be enough. It was not!RESPONSE: Strategies to address this included advocacy and in Rwanda ‘focal points’…
- #11: One of our biggest challenges currently is what is the mix of ensuring availability of SDM services versus promoting SDM as a new method. People need to know about SDM or they won’t ask for it. ‘Knowledge’ is often based on users’ experiences and whether SDM is discussed by friends, outside of clinics. We did not focus on demand creation during early phases – more than posters in clinics for the most part. This was a tactical error. This has become complicated, also, by the fact that governments and donors are promoting underutilized methods (and not SDM), so there is competition among FP methods, and those being promoted more heavily are, of course, more well known.Should we have used more resources for demand creation, to match promotion efforts of other method by other projects, donors, and governments? How much would be needed? We don’t have good answers, except that we should have done more early in scale up.
- #12: The other big issue is that once pilots are finished, it is almost impossible to get user information from partners working in the expansion phase, since we were not funding them. AND you need centralized user info to share with managers and policy makers to maintain political support.ALSO, new questions began to arise with expansion, eg, can women and men REALLY manage their white bead days?RESPONSE…
- #14: So, going back to our scale up waves, you can see that at different phases, different issues arose, many because we either assumed incorrectly something would happen or not happen, or miscalculated responses to SU efforts. And I think we understand better the importance of the running line at the bottom of the waves – the need to continue to advocate and communicate results, especially because scale up efforts take you to new partners at different levels and you cannot manage all situations during that process.