Find out more: Learning about evaluation with small cohorts: pilots to test the methodologies

—

Overview

The Most Significant Change (MSC) technique is a participatory, story-based method involving the collection and selection of significant change stories that have occurred in the field. Stories are usually elicited directly from programme participants. These stories are then passed upwards in the organisational hierarchy to panels of stakeholders who assess their significance, discuss how they relate to the wider implications of the changes and review the available evidence that supports them. This process helps reduce the number of stories to those identified as being the most significant by the majority of stakeholders.

While MSC was originally designed as an impact monitoring – rather than an evaluation – approach, it has since been adapted for use in impact evaluation by expanding the scale of story collection and the range of stakeholders involved. As an impact method, it is vulnerable to selection and social desirability bias. For these reasons, it is not typically used alone for impact evaluations, but usually precedes or complements summative evaluation and may best be used alongside more rigorous approaches, such as Contribution Analysis or Process Tracing, when tackling causal inference.

What is involved?

A detailed guide provided by Davies and Dart (2005) includes 10 steps, of which Steps 4, 5 and 6 are deemed fundamental while the remainder are discretionary.

    1. How to start and raise interest: Evaluators should be clear about the purpose of using MSC within the organisation. It may be useful to identify people excited by MSC who could act as catalysts in the process.
    2. Establishing ‘domains of change’: Domains of change are fuzzy categories defined to guide the significant change stories sought.
    3. Defining the reporting period: The frequency of collecting stories varies. Higher frequency reporting allows people to integrate the process more quickly but increases the cost of the process and risks the participants running out of identifiable SC stories.
    4. Collecting stories of change: Significant Change Stories can be captured in different ways including interviews and group sessions. Stories should be recorded as they are told and should be short and comprehensible to all stakeholders.
    5. Reviewing the stories within the organisational hierarchy: Storytellers discuss their stories and identify and submit the most significant ones to a level above. The same process is run at mid-levels, with stories selected and submitted to the next level. This process allows widely valued stories to be distinguished from those with only local importance. Depending on the evaluation aims and the scale of the project, the different levels may involve beneficiaries, field workers, managers, donors and investors.
    6. Providing stakeholders with regular feedback about the review process: Results must be fed back to storytellers.
    7. Implementing a process to verify the stories if necessary: Verification of the accounts can be beneficial.
    8. Quantification: While MSC is essentially qualitative, the quantification of surrounding information may be useful.
    9. Conducting secondary analysis and meta-monitoring: It may be useful to classify and examine the topics identified in SC stories using thematic coding.
    10. Revising the MSC process: MSC should not be used in an unreflective way; rather, the implementation should be adapted throughout the process.

Download a Most Significant Change case study here
Download a longer briefing on Most Significant Change here

 

Useful resources

A detailed description of the methodology and its application is available here:

Davies, R., and Dart, J. (2005) The ‘most significant change’ (MSC) technique. A guide to its use.