Report | Impact evaluation with small cohorts: Methodology guidance

_{17 June 2022}

Summary

Improving access and widening participation interventions in higher education are designed to produce a range of outcomes, and effective evaluation of such activities has assumed increasing importance over the last 15 years. However, the higher education sector has often struggled to generate effective and robust evaluation for these interventions.

Many interventions are not compatible with conventional forms of causal inference, such as systematic review, randomised controlled trial and quasi-experimental design, due to their complex nature, resource issues, emergent or developmental features, or small scale. 

Faced with these challenges, evaluators, service delivers and policymakers are increasingly seeking alternative understandings of causality to fill perceived gaps in actionable evidence. This report focuses on some of these alternatives, specifically designed for impact evaluation that can be used with small cohorts: so-called ‘small n’ impact evaluations.

This guide includes a discussion of the philosophies that underpin different causal strategies, the types of questions regarding cause and effect prioritised in different approaches and, thus, the nature of the evidence that will emerge from resulting studies. It then sets out a taxonomy of the various methodologies, before describing each one in detail.

Read the report