Introduction

When to evaluate?

An effective evaluation does not occur at the end of an intervention. It is an ongoing process that helps understand how the intervention is working, what effect it is having, and how it is influenced by internal and external factors. Although evaluation is generally more effective when built in from the beginning, evaluations can also be retroactively applied to existing or completed initiatives.

Types of evaluations

  • Implementation and Process Evaluation –  This type of evaluation provides information about how best to revise and improve activities. It can be used to assess whether the initiative, analysis, and underlying assumption(s) are being implemented as intended. It is often helpful for pilot projects and new services and schemes. Implementation and process evaluation can also be used to monitor the progress and delivery of ongoing initiatives.
  • Impact Evaluation – This type of evaluation is used to help decide whether a service or scheme should be adopted, continued, or modified for improvement. Initiatives are assessed at the end of an operating cycle. It is our expectation that over time the sector will have high-quality impact evaluations for all their out-/in-reach schemes. As an affiliate What Works Centre, TASO specifically promotes and generates rigorous experimental and quasi-experimental impact evaluation methodologies.

As outlined by the Office for Students (OfS), all evaluations funded or co-funded by Access and Participation Plans should have some element of impact evaluation. This is important in demonstrating that initiatives have the desired impact on student outcomes.

This guide – known as our Monitoring and Evaluation Framework (MEF) – promotes an impact-driven evaluation approach, interlaced with process evaluation, to determine which interventions work in transforming access and student outcomes.

The evaluation process

The MEF contains four main steps to consider when planning your evaluation – from establishing your Theory of Change in ‘Step 1: Diagnose’ to reporting on your findings in ‘Step 4: Reflect’.

The evaluation process is iterative. Your findings in each step help support continuous improvement. For example, the findings from ‘Step 3:Measure’ should feed into the refinement of your Theory of Change, research questions and methods in ‘Step 4: Reflect’.

The framework covers pre-16 outreach through to initiatives to address gaps in graduate outcomes. It is not intended to replace in-house evaluation support.

This next sections will guide you through the MEF step-by-step.