Improving the evaluation of integrated programs: A guide for multisector evaluation
Around the world, families living in poverty face an array of complex challenges to securing a better life. Challenges include limited access to a quality education, violence, weak governance, poor-quality health care and few opportunities to earn a living wage. These barriers do not operate in isolation: They are often interrelated and cannot be addressed with just one approach. That is why efforts to improve the lives of people in underserved communities need to better reflect the interconnected nature of those people’s experiences.
Achieving the Sustainable Development Goals requires new ways of thinking that break with the status quo. Recognizing this, FHI 360 has led a global conversation over the past three years about the power of integrated, multisector approaches. We define integrated development as an intentional approach that links the design, delivery and evaluation of programs across disciplines and sectors to produce an amplified, lasting impact on people’s lives. Measuring and demonstrating where integrated approaches best add value is one of the most pressing needs to continue moving this agenda forward.
Integration is not a new concept. Many integrated programs have been implemented and rigorously evaluated. A systematic review conducted by the FHI 360 integrated development team (manuscript in preparation) found more than 500 impact evaluations of integrated programs, which are all accessible and searchable online in FHI 360’s Integrated Development Evidence Map. But, less than 4 percent of those evaluations were designed to assess the added value of integration itself, and many evaluations noted a specific need for guidance on approaches and methods that are best suited for multisector programs. Clearly, there is a sizeable gap in the evidence base.
With this in mind, FHI 360 has recently released a comprehensive framework for evaluating multisector, integrated programs: Guidance for Evaluating Integrated Global Development Programs. This document, funded by a grant from the Bill & Melinda Gates Foundation, summarizes current research methodologies and approaches to better assess the nuanced nature of complex, integrated models. The framework also addresses key considerations on formative research, performance indicators, program monitoring, process evaluation, cost analyses, impact evaluation and scale-up evaluation. The authors intend the guidance to be flexible so users can adapt it to their particular circumstances.
Evaluations that have used these approaches have produced exciting findings that add to the evidence base on integrated programs. One example is FHI 360’s Uganda Community Connector project, which took a participatory approach to developing indicators, thinking through the impact pathway of the activities so that the added value of integration could be measured. The household-level measures, called the “CC See10,” were able to measure improvements in women’s economic and social empowerment in households, which was an expected outcome of the integrated activities. Thoughtful consideration and planning of intended outcomes, beyond just improved outcomes, is a simple but powerful tool. The guidance document explores these approaches and considerations, as well as many others.;
This guidance will add to the continuing conversation on evaluating integrated approaches at the American Evaluators Association Conference in October 2016. Moving forward, more programs need the freedom to use evaluative approaches that are suitable for integrated programs so that we can better understand when to use integrated approaches.
If you have questions or comments, please contact FHI 360’s Integrated Development department at askID@fhi360.org.
Photo credit: Tessa Ahner-McHaffie/FHI 360