New report: harnessing Theory of Change to tackle ethnicity degree awarding gaps
We worked with six higher education providers in the UK to explore the use of Theory of Change to better understand their interventions.
The ethnicity degree awarding gap – the difference in the proportion of degrees classed at a 2:1 and above awarded to ethnic minority students compared to white students – is an established problem across higher education in the UK.
To reduce the longstanding ethnicity degree awarding gap across the higher education sector, it is important that providers better understand how their interventions intend to work, and how short- and intermediate-term outcomes, such as belonging, may link to the gap.
Previous TASO research (PDF) found that although many higher education providers have plans which aim to address the gap, the majority did not include a theory of change for their proposed interventions. A theory of change helps describe how activities link to desired outcomes. The research also highlighted a widespread lack of confidence in evaluating interventions that aim to reduce the gap.
The project
TASO commissioned an evaluation team from Staffordshire University and Advance HE to work with Birmingham City University, Loughborough University, Manchester Metropolitan University, University of Law, University of Southampton and the University of York.
Workshops were held with each provider to produce a Theory of Change and associated evaluation plan for their interventions to address the ethnicity degree awarding gap. The interventions ranged from assessment guidelines and early intervention, through to relationship building and dissertation retreats.
Key learnings and recommendations
The project led to several key recommendations and learnings:
- The maturity of an intervention impacts theory of change development but regardless of the life-stage of an intervention, the process of developing a theory of change and actively articulating ideas, plans, and theories is beneficial for key stakeholders to understand and unpick an intervention.
- Face-to-face theory-of-change development may be more effective initially. Where possible, and especially at the start of the theory-of-change journey, having stakeholders come together in a shared space to flesh out the content simplifies the process and reduces the amount of time required.
- Incorporate diverse representation in theory-of-change and evaluation plan development. It is important to involve a range of stakeholders to ensure the outputs consider different perspectives. This could include representation from students, senior staff, evaluators, data teams, strategic planners, operational staff, and intervention planners/developers.
- Recognise the time requirements for developing a theory of change. Theory of change development is not designed to be completed quickly, but as an iterative process that takes time and thought.
- Critical questions can support the development of a theory of change. Questions – such as How does this activity lead to that outcome? Do we really think this outcome will happen as a result of the intervention? – can help during development to focus discussion and support completion.
Understanding what works
By providing example Theories of Change and evaluation plans, this project helps in the quest for more evidence on what works in closing equality gaps in higher education. It is vital that we effectively evaluate interventions to address the awarding gap, and Theory of Change is a key first step in this journey.
We hope that the insights from this project and the associated outputs will be beneficial in supporting the sector to tackle this significant inequality.