Impact assessment based on self-reported attribution in complex contexts of rural livelihood transformations in Africa.

点击翻译按钮获取中文摘要

There is a growing demand for impact evidence within development agencies, both to support their own learning and to account responsibly to other stakeholders. Internal M&E systems are an important foundation for such assessment but it is widely recognised that they can usefully be complemented by periodic independent impact assessment (IA). However, doubts remain over the reliability and/or cost-effectiveness of much IA, prompting ongoing experimentation and academic debate over alternative approaches.The proposed research addresses the particularly difficult issue of how to assess the impact of development activities intended to benefit poor men and women caught up in complex processes of rural transformation in Africa. The core issue addressed is how to attribute change specific interventions in a way that is reliable, timely and cost-effective to different stakeholders, without at the same time distorting or constraining the development activity being assessed. While the impact attribution problem is pervasive in development practice, the research will be focus particularly on the work of two NGOs: Self Help Africa (SHA) and Farm Africa (FA), and their ongoing activities in Malawi and Ethiopia. More specifically the research will assess scope in such contexts for rigorous and cost-effective impact assessment based on (a) systematic monitoring of economic security at the household level combined (b) with in-depth interviewing designed to elicit self-reported attribution by respondents. This approach can be contrasted with both participatory approaches to impact assessment, and those that rely on statistically inferred attribution using either randomised control trials or quasi-experimental methods. The proposed research comprises three complementary strands. Strand 1: four baseline studies (two in Malawi, two in Ethiopia) followed by two rounds of programme monitoring studies conducted by the SHA and FA with specialist technical support from the NGO Evidence for Development (EfD). This will draw upon and enhance EfD's pioneering work in developing the Integrated Household Method (IHM) for measuring and modelling the reslience of household level cash income and food consumption. Strand 2: two rounds of impact studies by independent consultants, using a qualitative individual impact protocol (QUIP) that combines Strand 1 monitoring with semi-structured interviews based on a self-reported attribution methodology.Strand 3: two rounds of meta-evaluation, through which the quality of evidence obtained through Strands 1 and 2 will be reviewed, and additional evidence collectied of how it was interpreted and utilised by different stakeholders, and how it affected their working relationships. A particular focus here is on what evidence obtained through Strand 2 added to that obtained from Strand 1 and whether this justified the extra cost.The research will thereby contribute to improving feedback loops in support of improved development interventions to support rural livelihood transformation in Africa, with potential methodological benefits in support of aid effectiveness more generally. In addition to the core methodological goals of the research, the research will also address immediate and substantive questions facing the two participating NGOs in their work in Ethiopia and Malawi, relevant also to other organisations carrying out similar activities. More specifically, SHA and FA are seeking to improve their understanding of what combination of interventions are most effective in improving the economic and food security of poor rural households in the two countries. Activities to be investigated include promoting improved seeds and cultivation practices, animal husbandry, supply chain upgrading, natural resource management, microfinance, and diversification into non-farm activities.