A Review of Recent Developments in Impact Evaluation
Author: Asian Development Bank
Size: 84 pages (1.1MB)
How can impact be credibly attributed to a particular intervention? This report discusses the merits and limitations of various methods and offers practical guidance on impact evaluation. A rigorously conducted impact evaluation produces reliable impact estimates of an intervention through careful construction of the counterfactual using experimental or non-experimental approaches.
The impact evaluation literature has been growing rapidly. While most studies focus on education and health, other sectors such as infrastructure, agriculture, and microfinance have seen a rapid increase in both the number and rigour of evaluations.
An impact evaluation should indicate more than whether an intervention works or not. It should also indicate in what context the intervention does or does not work, and to what extent its impacts vary across different groups of beneficiaries.
The core challenge lies in credible attribution of the observed changes in the outcome variables to the intervention being evaluated. The difficulty of attribution arises because the intervention is not randomly assigned or individuals do not participate in the programme in a random manner. Simply comparing those 'treated' by the intervention and the untreated will result in biases in evaluation results and lead to misleading conclusions about the impact of the intervention.
Different methods based on observational data have been developed to overcome this attribution challenge in impact evaluation:
There is no single well-developed evaluation design that can universally be applied to all projects. The optimal design for a project varies across and within sectors. Further lessons include the following:
Asian Development Bank, 2011, 'A Review of Recent Developments in Impact Evaluation', Asian Development Bank, Manila, Philippines
Organisation: Asian Development Bank (ADB), http://www.adb.org/