Blog21 May 2026
In 2023, we published guidance on ‘small-n’ methodologies to help address the challenges many providers face in establishing the causal impact of interventions with small numbers of participants. Since then, we have run a series of pilots and commissioned two evaluations of wellbeing interventions using a range of these methodologies.
To describe these methodologies more accurately, and to align with the academic literature, we are now using the term ‘theory based evaluation’ instead of ‘small n’. This is because theory-based evaluation describes the type of evaluation, rather than the size of the group to which it is applied. Our guidance will remain unchanged, but this terminology better reflects the broader range of contexts in which these methodologies can be used.
A problem of language
In our interactions with providers, we noticed that the terms ‘small-n’ or ‘small cohorts’ are often conflated with smaller provider settings. These are two separate – but easily confusable – terms. Small providers are providers with a small number of students. Small cohorts are small groups within any size of provider. This is another reason we decided to switch to using ‘theory-based evaluation’: to distinguish these methodologies from evaluation approaches specifically designed for small providers.
Theory-based evaluations provide an alternative way of evaluating impact when experimental designs – such as randomised controlled trials and quasi-experimental evaluation – are not feasible, for example because there are too few participants to establish a comparison group. However, these methodologies are not only confined to use with small cohorts – they can be used for larger groups and in other contexts.
So what are theory-based approaches – and when can I use them?
As our guidance outlines, theory-based approaches are a range of methodologies that can be employed and combined to ask not only whether an intervention worked, but also how and why.
Theory-based approaches can be used both when working with small cohorts and with larger systems-change interventions such as whole-provider approaches.
They can:
- work with only a small number of cases (even one)
- help untangle relationships between multiple causal factors in complex systems
- be used with emergent interventions that are still changing and developing
- sometimes be applied retrospectively after a programme has finished.
In the absence of comparator groups that help to establish causality, theory-based approaches examine how an intervention is expected to produce change, mapping this through clear causal pathways. These pathways are tested within the intervention context.
Different theory-based methods have distinct ways of investigating and explaining the relationships between the outcomes, contexts and assumptions in a theory of change. In realist evaluation, for example, evaluators use the theory of change to develop causal pathways known as context-mechanism-outcome clusters that inform questions for the data collection process.
Theory-based evaluations therefore look at how the intervention works in that particular context, and how the different contributing factors relate to the outcomes.
Theory-based approaches are acknowledged in the Magenta Book as a form of impact evaluation that can establish causality, though they may not be able to establish precise effect sizes of impact, and the findings may not be generalisable beyond the participants in the evaluation.
Does small cohort mean small institution?
While theory-based methods are well-suited for establishing causal impacts with small cohorts, these can be small cohorts in any size of institution. They can also be applied to larger groups, in cases where it is not possible to have a comparator group.
However, theory-based evaluations should not be mistaken as smaller or less resource-intensive evaluations. As our recent work on theory-based evaluations has shown, they often require mixed-method and qualitative data collection, which can be time-consuming. Some methodologies may also require evaluators to have the necessary skills in developing detailed theories of change, data collection and analysis. It is therefore recommended to allocate sufficient time, skill and resources to conduct them well.
Where can I learn more?
Access further guidance:
- Theory-based evaluation methodologies guidance
- Webinar on theory-based evaluation methodologies
- Pilot projects testing the use of theory-based methodologies in higher education
- Report on wellbeing interventions evaluated using theory-based methods