Lucy Hodson, Director of Planning and Intelligence; Philippa Try, Data Analyst (Evaluation); Myles Payne, Data Analyst (Evaluation), Planning and Performance Department, Birmingham City University

5 March 2024

Mind the (ethnicity degree awarding) gap: A changed way of seeing Theory of Change development

Birmingham City University (BCU) is a large and diverse university with a focus on practice-based learning. At BCU, we are dedicated to developing widening participation interventions that have meaningful impact on the lives of our students and we jumped at the opportunity to be a part of the ethnicity degree awarding gap project with TASO.

BCU joined Wave 1 of the Office for Students new approach to access and participation plans (APPs), producing a plan for 2024–25 to 2027–28. As seen across the sector, there is considerable work to be done to improve the outcomes of students from minority ethnic groups. BCU’s significant ethnicity degree awarding gap (that is the difference in the proportion of degrees classed at a 2:1 and above awarded to ethnic minority students compared to white students) is a focus of the new APP.

This aligns with an even tighter focus on the gap above and beyond the confines of the APP set by incoming Vice-Chancellor, Professor David Mba, who joined BCU in October 2023. TASO’s project was an excellent opportunity to develop a consistent approach to intervention design and evaluation to ensure that our work not only has meaningful impact for our students, but also provides the sector with evidence of ‘what works’, with research and evaluation outputs linked to our APP in a consistent and accessible manner.

Facilitating conversation with a shared roadmap and language

BCU was already accustomed to Theory of Change, including use of resources such as ChangeBusters from Advance HE. However, using TASO’s Enhanced Theory of Change and associated evaluation plan helped us develop a structured and detailed design philosophy.

The knowledge gained from the project workshops helped facilitate conversations internally, engaging senior management, students, grassroots provision and all in between. This allowed for stakeholders across a range of roles and responsibilities to engage effectively and collaboratively from a shared conceptual foundation.

In a sector where every institution or funding body has their own version of the same set of acronyms, having a consistent language and approach allowed for meaningful engagement during the creation process. It also allowed for the creative development of interventions including a roadmap clearly identifying assumptions and change mechanisms that keep projects on track and aid evaluation processes.

Supporting evaluation and outlining assumptions

Evaluation can seem like a daunting affair, no matter the size or shape of an institution, and when the specific context is key to an intervention, this has the potential to limit transferability of evaluation findings. With this in mind, it’s important to flag that our Enhanced Theory of Change is a living and accessible document, which has already undergone multiple iterations. It will be reviewed and updated where necessary during each stage of our intervention rollout – producing accessible assessment principles for all of our courses.

It has been highly beneficial to outline assumptions clearly, especially for our more complex interventions. It has answered questions about why certain activities have been selected, and moved discussions towards understanding causality and the impact of activities.

From activities to impact

Theory of Change links resources and activities to the impact they intend to generate, starting with identification of long-term goals and associated outputs and outcomes. When used to ground decision-making and resource allocation in higher education, it provides an accessible and shared foundation for change management, identifying all that is necessary for impactful goals to be achieved. With institution-wide buy-in to the approach, measurable progress becomes possible, especially when evaluation is built in from the outset.

The opportunity to challenge assumptions, be agile in response and evidence progress towards longer term goals, ensures transformation programmes are comprehensively rationalised.