Project | Learning about evaluation with small cohorts: pilots to test the methodologies

About the project

To evaluate impact of interventions, TASO promotes the use of rigorous methodologies. However, these traditional methods may not be suitable for interventions with small cohorts of participants. TASO evaluation guidance, impact evaluation with small cohorts, is designed to help higher education providers overcome these challenges.

The guidance sets out eight evaluation approaches suitable for use in small-cohort contexts. This project involved six higher education provider teams testing four of these evaluation methods: Realist Evaluation, Contribution Analysis, Most Significant Change and Qualitative Comparative Analysis. The six provider teams and their reports, case studies and Theory of Change documents are listed below.

The pilot project teams identified the challenges that arose during the implementation of these methodologies, the benefits produced by small-cohort evaluation approaches, and the lessons learned from the pilot projects. 

A series of recommendations were developed for those higher education providers looking to evaluate small-cohort interventions.

Related resources

Background

At the heart of impact evaluation is the need to establish a connection between cause and effect in order to explain how and why activities lead to changes in desired outcomes. While traditional experimental and quasi-experimental methods can provide the higher education sector with the confidence required to state causal inference, it is not always possible to use these methods, particularly for small and specialist providers.

The higher education sector needs to use a wider range of impact evaluation methodologies. This project aimed to develop and test a range of evaluation designs that could be used to assess the impact of small-group interventions.