Improving development aid design and evaluation: Plan for sailboats, not trains

Rachel Kleinfeld
2015

Summary

How do reforms that require political engagement differ from traditional technical reforms? Why is political engagement different, and what are the implications for design and evaluation? How should development programmes that engage politics be designed? And how can those who fund or implement such programmes evaluate whether their efforts are contributing to reform? This report seeks to address these questions.

The report is concerned with large-scale political and policy engagement. It applies to the subset of the development world that is engaged in democratisation. But it also has a broader ambit. The development community has more recently become involved in governance, anticorruption, transparency, and rule of law programmes. These efforts universally affect laws and policies, and nearly all face opposition — and thus all are political.

Key findings:

  • Successful efforts in political and social change do not follow the path of a train headed down a track. They are not about travelling a predetermined route toward a preset best-practice goal along a timetable of benchmarks and chalking up incremental victories. Nor can they be measured based only on whether one’s policy goal has been met. Reform efforts must be adaptive and iterative to test assumptions and counter equally adaptive opponents. Like sailboats, they must use the wind of opportunity when it arises, and expect that they will move sideways at times to get to their end goals.
  • In more political development programmes, opponents may contest both ends and means. Programmes that get adopted are rarely technical best practices, but rather those that amass the most political support. The presence of opposition actors means that reforms are frequently followed by counter-reforms. Change swings back and forth. Measuring success at only one point in time means little for whether a reform will be sustained.
  • Political variables are interdependent, but popular measurement tools such as regressions and randomised controlled trials assume variables can be separated. These techniques can determine which interventions are most effective—but not how to get those programs implemented. Designing programmes that alter the underlying rules of political and social systems is the key to successful reform.

Recommendations:

  • Design programmes and funding to anticipate counter-reforms and multiple battles. Opposition learns, too: techniques that worked at one point may fail at another.
  • Engage local partners who can amass broad coalitions. Avoid making groups overly obligated to donor agendas that can cost them local support.
  • Ensure flexibility for programmes and budgeting, and expect changes. Test hypotheses throughout a programmes’s life cycle. Design contracts to enable closing projects and moving funds among projects so that acting on what works does not carry a stigma or lead to perverse incentives. Measure programmes based on whether they have created long-term, broad coalitions and/or elite influencers with real political power who are growing stronger.
  • Prepare for windows of opportunity before they open. Invest in coalitions, policy development, and social networks ahead of time.
  • Determine whether programmes have shaped the rules of the system to make change easier. Programmes that enable organising, increase transparency and public voice in policy, reduce violence against reform advocates, and increase avenues to power are types of systemic changes that allow reform.
  • Measure reform based on the space of the possible. Look at all the potential options in a policy space, including possible counter-reforms, not just the currently ascendant policy.

Source

Kleinfeld, R. (2015). Improving Development Aid Design and Evaluation: Plan for Sailboats, Not Trains. Washington, DC: Carnegie Endowment for International Peace.