Monitoring and evaluation

Participatory tools and approaches

Page contents

• The Case for Participatory M&E
• Challenges in Using Participatory Approaches
• Participatory Tools: Case Studies
• Participatory Rural Appraisal
• Key Informant Interviews
• Most Significant Change Technique
• Outcome Mapping

Monitoring and evaluation has typically been led by outside experts, measuring performance against pre-set indicators and using procedures and tools designed without the participation of key stakeholders such as the programmes’ intended beneficiaries. Evaluations in particular, because they are very often conducted by external consultants, can be seen as a form of control.

There is widespread recognition that M&E should take a more inclusive, participatory approach. Participation in this sense means stakeholders are involved in deciding how the project or programme should be measured, in identifying and analysing change, and in acting on results. Nevertheless, there are few empirical studies of the effectiveness, quality and objectivity of participatory approaches. How to operationalise participation and which methods work in which contexts is the subject of ongoing debate.

The Case for Participatory M&E

Proponents of participatory monitoring and evaluation (PM&E) argue it is more ethical, cost-effective and accurate in determining the effects of interventions on people’s lives than conventional approaches. Participation in decision-making processes can also motivate people to want to see those decisions implemented effectively. Another motivation for participatory M&E is to strengthen organisational and institutional learning.

Some argue that participatory approaches can generate both quantitative and qualitative data equally effectively.

Chambers, R., and Mayoux, L., 2003, ‘Reversing the Paradigm: Quantification and Participatory Methods’, Enterprise Impact, London
What role should participatory methods play in assessing the impact of development activity? A common assumption is that rigorous quantitative data can only be generated by questionnaire surveys or scientific measurement. Another is that participatory methods can only generate qualitative insights.  This paper from the Enterprise Development Impact Assessment Information Service (EDIAIS) discusses experiences and innovations which show these assumptions to be false. It argues that participatory approaches can generate accurate qualitative and quantitative information and should form the basis for monitoring, evaluation and impact assessment.
Access full text: available online

Challenges in Using Participatory Approaches

Whilst the ideological case for participation is widely acknowledged, participatory M&E is not without its critics. Crucially, labelling M&E as ‘participatory’ doesn’t necessarily guarantee that all stakeholder groups have participated, and there are often issues around who participates and who is excluded from these processes. Subsequently, the representatively of the findings and recommendations of participatory evaluations have been criticized.

Gregory, A., 2000, ‘Problematizing Participation: A Critical Review of Approaches to Participation in Evaluation Theory’, Evaluation, Vol 6, no. 2, pp. 179–199
It is widely accepted that evaluation is a social process which implies the need for a participatory approach. But what is understood by 'participation'? This critical review from Hull University Business School argues that the blanket use of the term has masked the heterogeneity evident in its realisation in practice and highlights a lack of transparency in participatory methods in evaluation.

Operationalising participatory M&E can also be problematic. There is a need to be sensitive to the local socio-economic and political situation, and consider under what conditions PM&E approaches can be used without increasing the vulnerabilities of already marginalised groups.

Estrella, M., 2000, ‘Learning from Change: Issues and Experiences in Participatory Monitoring and Evaluation’, International Development Research Centre
Since the 1980s concepts of participatory monitoring and evaluation (PM&E) have entered the policy-making domain of larger donor agencies and development organisations. This introductory chapter from Learning from Change: Issues and Experiences in Participatory Monitoring and Evaluation draws on twelve case studies to describe how different stakeholders have applied PM&E approaches across a range of purposes and contexts. It outlines some of the key concepts and differences between participatory and conventional approaches to M&E and highlights some emerging issues.
Access full text: available online

Vernooy, R., Qui, S., and Jianchu, X., 2006, ‘The Power of Participatory Monitoring and Evaluation: Insights from South-West China’, Development in Practice, Volume 16, Number 5, pp. 400-411
Participatory monitoring and evaluation (PM&E) is a partnership between researchers and other stakeholders to systematically assess research or development activities. Focusing on participatory field research for community-based natural-resource management (CBNRM), this article from Development in Practice describes the capacity-building experiences of two research teams in Yunnan and Guizhou provinces in south-west China. The ongoing democratisation and decentralisation processes in China aim to allow more space for local voice and decision-making power over NRM. So, who participates and what difference does participation make?  
Access full text: available online

Case Study: Participatory Monitoring and Evaluation of the Zambia Social Investment Fund (ZAMSIF)
This case study provides an example of how to set up a participatory monitoring and evaluation framework.

ITAD, 2004, ‘Participatory Monitoring and Evaluation: Documentation to Support Generic Framework’, ITAD, London
The Participatory Monitoring and Evaluation (PM&E) consultancy for the Zambia Social Investment Fund (Zamsif) aims to establish sustainable PM&E processes at both the community and district levels. This report from ITAD and RuralNet Associates discusses the development and rationale of the PM&E framework chosen for the project. It constructs detailed diagrams to model the consultancy’s work and explains current and potential uses of the framework.
Access full text: available online

Participatory Tools: Case Studies

Despite growing interest in the subject, there is no single definition or methodology of PM&E, and it encompasses a wide range of tools and approaches. Common among these approaches are values such as shared learning, democratic processes, joint decision making, co-ownership, mutual respect and empowerment. Below are some examples of participatory tools and how they have been used in practice.

Participatory Rural Appraisal (PRA) encompasses a broad range of methods to enable local people to analyse their own realities as the basis for planning, monitoring and evaluating development activities. PRA uses group exercises to facilitate information sharing, analysis, and action among stakeholders.

Chambers, R., 2007, 'From PRA to PLA and Pluralism: Practice and Theory', IDS Working Paper, no. 286, Institute of Development Studies, Brighton
How have the theory and practice of participatory methodologies in development activities changed since the mid 1970s? What variants and applications of these methodologies have emerged? This paper from the Institute of Development Studies (IDS) traces the spread of Participatory Rural Appraisal (PRA) and Participatory Learning and Action (PLA), and identifies strengths and weaknesses in the ways they have been implemented in development contexts. Whilst inflexible applications of PRA and PLA may produce disappointing results, when executed with spontaneity and creativity, these approaches can be a source of community empowerment.
Access full text: available online

Key informant interviews are a rapid assessment methodology which can be used as an intermediate indicator of outcomes as an alternative or supplement to full impact assessments.

Price, N., and Pokharel, D., 2005, ‘Using Key Informant Monitoring in Safe Motherhood Programming in Nepal’, Development in Practice, Volume 15, Number 2
The Nepal Safer Motherhood Project (NSMP) works to improve maternal health and contribute to programme development at district and national level. This article from Development in Practice discusses the project’s use of Key Informant Monitoring (KIM). KIM is an adapted version of the peer ethnographic research method. Data is collected by community-based Key Informant Researchers (KIRs) and used for monitoring and planning. KIRs have proved useful sources of information and acted as change agents by spreading safer motherhood messages.
Access full text: available online

The most signficant change technique involves the collection of significant change stories emanating from the field level, and the systematic selection of the most significant of these. These selected stories are then discussed and critically reflected on to help determine the impact of the development program or activity.

Davies, R., 2005, ‘The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use’, MandE, London
This paper from MandE News, an online monitoring and evaluation news service, outlines an innovative qualitative monitoring technique known as the 'most significant change' (MSC) approach. The MSC technique is a participatory method of collecting and analysing stories from the field which focuses on monitoring intermediate outcomes and impact. It provides a simple means of making sense of a large amount of complex information and is best suited to large-scale, open-ended projects which would otherwise be difficult to monitor easily using traditional methods. 
Access full text: available online

Case Study: Cabungo in Malawi
There is evidence that the most significant change technique can enhance organisational learning and performance.

Wrigley, R., 2006, ‘Learning from Capacity Building Practice: Adapting the ‘Most Significant Change’ (MSC) Approach to Evaluate Capacity Building’, INTRAC Praxis Paper no. 12, International NGO Training and Research Centre, Oxford
There is growing recognition of the need to take a multi-stakeholder approach to evaluation, which promotes local ownership and builds capacity for reflection, learning, improved performance and self-determination. This paper from the International NGO Training and Research Centre (INTRAC) reflects on the use of the 'Most Significant Change' (MSC) methodology to evaluate the capacity building services of CABUNGO, a local capacity building support provider in Malawi.
Access full text: available online

Outcome Mapping is an alternative to theory-based approaches to evaluation that rely on a cause–effect framework; rather, it recognizes that multiple, nonlinear events lead to change. It focuses on people and changes of behaviour and how far development interventions have built the capacity of the local community. Outcome mapping assumes only that a contribution has been made, and never attempts attribution.

Earl S., Carden, F., and Smutylo., T., 2001, ‘Outcome Mapping: Building Learning and Reflection into Development Programs’, International Development Research Centre, Canada
Development organisations are increasingly under pressure to demonstrate that their programmes result in positive changes. This paper from the International Development Research Centre (IDRC) argues that impacts are often the product of events for which no single agency can claim full credit. Outcome mapping moves away from impact assessments to focus on changes in the behaviour of the people it works with directly.
Access full text: available online

New additions