Monitoring and evaluation

Monitoring poverty reduction strategies

Page contents

• Poverty Monitoring Systems
• Lessons Learned in Design
• Indicators
• Institutions for PRS Monitoring
• What Role for Donors?
• Civil Society Participation
• Strengthening National Statistical Capacity

Progress towards national poverty reduction goals set out in Poverty Reduction Strategies (PRS) is monitored and assessed by governments, donors and civil society to determine the effectiveness of government expenditure, guide aid allocations, or inform and stimulate national debate on development progress. However, PRS monitoring has historically suffered from underinvestment, lack of co-ordination and weak national capacity.

There is considerable debate about how to overcome these constraints, which institutional arrangements for monitoring work best, and how different actors can contribute to the process.

Poverty Monitoring Systems

The institutions, relationships and systems established to monitor poverty reduction are often referred to as the ‘Poverty Monitoring System’. Underinvestment in these systems has arguably resulted in a major shortfall in understanding the effectiveness of poverty reduction efforts.  

Holvoet, N., and Robrecht, R., 2006, 'Putting the New Aid Paradigm to Work: Challenges for Monitoring and Evaluation', Instituut voor Ontwikkelingsbeleid en-Beheer, Antwerp
What are the challenges in monitoring and evaluating Poverty Reduction Strategy Paper (PRSP) outcomes? Have issues been neglected in these key areas? This study by the Institute of Development Policy and Management, University of Antwerp, reviews and assesses M&E systems for 11 Sub-Saharan African countries. These systems are embryonic in the countries reviewed, and are a weak link in most PRSPs. The emphasis on participation and comprehensiveness places unrealistic demands on national capacity, exacerbated by the scant attention given to M&E by donors.
Access full text: available online

Lessons Learned in Design

Setting up a poverty monitoring system means defining goals, indicators and targets, agreeing data requirements and deciding who will collect data and how. In reality, the different components that make up the monitoring system can have independent origins, and there is often little coherence across the system.  

World Bank, 2002, ‘Chapter 3: Monitoring and Evaluation’, in PRSP Sourcebook, World Bank, Washington
How do we know if a poverty reduction strategy is effective? First, a poverty monitoring system is needed to track key indicators over time and space and to determine if they change as a result of the strategy. Second, rigorous evaluations should be done selectively to assess the impact on poverty of interventions that are key components of the strategy. This chapter from the World Bank's Poverty Reduction Strategy Paper Sourcebook examines the features of poverty monitoring systems and explores some of the key issues which arise during implementation.
Access full text: available online

Lucas, H., Evans, D. and Pasteur, K., 2004, ‘Research on the Current State of PRS Monitoring Systems’, IDS Discussion Paper no. 382, Institute of Development Studies, Brighton
What are the key factors that determine the success or failure of Poverty Reduction Strategies (PRS) monitoring efforts? Who should be involved? How should the information gathered be put to use? This study by the Institute of Development Studies reviews the main issues arising from the implementation of PRS monitoring systems. It draws on the experience of different countries and suggests possible monitoring approaches to suit specific contexts.
Access full text: available online

Overseas Development Institute, 2002, ‘Good Practice in the Development of PRSP Indicators and Monitoring Systems’, ODI, London
How can poverty reduction strategies best be monitored? And how can monitoring lead to greater success in reducing poverty? This study by the Overseas Development Institute (ODI) draws conclusions about best practice from a review of Poverty Reduction Strategy Papers (PRSPs) and suggests ways in which monitoring can be improved. PRSP monitoring calls for fresh thinking. It needs to be geared to what is new and challenging about the PRSP initiative – particularly the effort to engage a wider range of stakeholders in policy dialogue about poverty reduction at the national level. It also needs an understanding of the relevant policy processes and the possible uses of information in enforcing new kinds of accountability and learning about poverty reduction.
Access full text: available online

Indicators

Indicators are the quantifiable measures of progress towards the intended inputs, outputs, outcomes and impacts of a project, programme or strategy. They are the measures for assessing the quantitative and qualitative impact of development efforts.

Reaching collective agreement on indicators is a complex task, not least because positive and negative changes (in relation to project objectives and real world outcomes) have varying time frames of measurement/assessment, and developing credible short-term indicators of long-term change can be problematic.

Arguably the most important criteria for selecting indicators is that they are realistic - it is essential that the necessary time, resources and data are available to measure them. It is also widely acknowledged that indicators should be independent of possible bias of the observer, and fulfil the SMART criteria - ‘Specific - Measurable - Achievable - Relevant – Timebound’. 

World Bank, 2004, ‘Poverty Monitoring Guidance Note 1: Selecting Indicators’, World Bank, Washington
Access full text: available online

Indicators can only be used effectively to promote development outcomes if they are fully understood by, and command widespread support among, a broad range of national stakeholders.

Scott, C. and Wilde, A., 2006, 'Measuring Democratic Governance: A Framework for Selecting Pro-poor and Gender Sensitive Indicators', United Nations Development Programme, Oslo
How should we measure democratic governance? Most indicators are developed by external stakeholders to compare nation states and are not designed to help countries undertake governance reforms. This UNDP guide presents a framework for generating pro-poor gender sensitive indicators to help policy makers monitor and evaluate democratic governance at the country level. It argues that indicator selection is itself a governance process.
Access full text: available online

For guidance on selecting and using indicators in specific sectors, see:

•  GSDRC Guide to Measuring Governance
• GSDRC Guide to Rights Monitoring and Indicators
• GSDRC Guide to Quantifying Social Exclusion

Institutions for PRS Monitoring

Poverty monitoring systems typically involve a broad range of actors, including data producers, analysts and users from government, donors, consulting firms, think tanks and civil society. In this environment, a clear, coherent system of roles, responsibilities and reporting mechanisms is necessary to avoid fragmentation and ensure that information flows effectively back in to policymaking.

Bedi, T., et al., 2006, 'Beyond the Numbers: Understanding the Institutions for Monitoring Poverty Reduction Strategies', World Bank, Washington DC
Monitoring systems are central to the effective design and implementation of a Poverty Reduction Strategy (PRS). Many existing PRS monitoring systems lack coordination and a coherent institutional framework linking monitoring and decision making. This World Bank report draws on 12 country studies to conclude that PRS monitoring systems should build on existing elements to begin a process of gradual change. Clearly defined relationships, incentives and activities and identification of entry points in decision-making processes facilitate the supply of monitoring information and its integration into improving PRS policies.
Access full text: available online

Case Study: The Poverty Monitoring System in Uganda
The Uganda Poverty Monitoring System has seen some success in improving country ownership, increasing results orientation and promoting partnership.

Booth, D., and Nsabagasani, X., 2005, ‘Poverty Monitoring Systems: An Analysis of Institutional Arrangements in Uganda’, Overseas Development Institute, London
Monitoring activities can play an important role in promoting country ownership of poverty-reduction policies where they are closely related to a politically supported and maturing budget process. Focusing on Uganda, this paper from the Overseas Development Institute (ODI) is one of a series of country studies intended to inform the design and implementation of poverty monitoring systems (PMS). While Uganda had an early start in creating a political environment that enabled results to influence policy, problems with incentives and partnership continue. There will inevitably be setbacks in developing a PMS - a process that requires coordination, policy-relevance and strong donor-country partnerships.
Access full text: available online

What Role for Donors?

PRS monitoring has recieved criticism for being a process primarily set up to meet donor accountability requirements and support conditionality. Pressure for effective monitoring systems largely comes from donors, rather than from domestic stakeholders. There is considerable debate about what role donors can usefully play in supporting the monitoring process, in particular how they can adopt a country-led approach whilst meeting their own accountability requirements.

Lucas, H., and Zabel, M., 2005, ‘The Role of Donors in PRS Monitoring Systems: Advantages and Disadvantages of the Different Forms of Involvement’, Report prepared for the World Bank, HLSP Institute, London
Access full text: available online

Civil Society Participation

Civil society has an important role to play in terms of overseeing and helping to hold governments to account for progress towards national poverty reduction goals. Nevertheless, civil society organisations often face significant obstables in systematically monitoring poverty reduction strategies where there is a weak culture of accountability or restricted access to information. How civil society participation and co-ordination can be improved is a key concern for civil society and donors alike.

Alliance Sud, 2006, 'Civil Society's Perspective on their Involvement in PRSP Monitoring: Assessing Constraints and Potentials in Nicaragua', Swiss Alliance of Development Organisations (Alliance Sud), Bern
What are the constraints and potentials facing civil society participation in Poverty Reduction Strategy (PRS) monitoring processes in Nicaragua? This Alliance Sud study reports on an ongoing initiative to develop an analytical tool to assess civil society’s participation in PRS monitoring processes from their own perspective. It finds that basic conditions for a significant role for civil society organisations (CSOs) in the monitoring system in Nicaragua are not met.
Access full text: available online

Strengthening National Statistical Capacity

Good statistics are vital for the effective monitoring of development programmes and strategies, and ultimately to support evidence-based policymaking. However, many developing countries lack the institutional capacity or effective systems for gathering data. Many existing statistical systems were predominately designed to meet immediate rather than long-term data needs, and therefore lack co-ordination.

There is increasing recognition of the need for a strategic approach to statistical capacity development, particularly following the Second International Roundtable on Managing for Development Results in February 2004 and the resulting Marrakech Action Plan for Statistics (MAPS). This emphasised the need to develop National Strategies for the Development of Statistics (NSDSs).  

Paris21 Secretariat, 2004, ‘A Guide to Designing a National Strategy for the Development of Statistics (NSDS)’, Prepared by the Partnership in Statistics for Development in the Twenty-first Century (PARIS 21)
There is increasing awareness of the need to strengthen statistical capacity to support the design, monitoring and evaluation of national development plans.  National Strategies for the Development of Statistics (NSDS) are designed to achieve this goal. This guide prepared by the Partnership in Statistics for Development in the Twenty-first Century (PARIS21) aims primarily to assist developing countries to design their NSDSs but will also be useful to development partners.
Access full text: available online

New additions