Annual Conference: How to Evaluate – Agenda
- Day 1 programme (7 May)
- Day 2 programme (8 May)
Day 1 programme (7 May)
10:00am Opening and welcome remarks
10:10am Keynote: evaluation, evaluation, evaluation
Speakers:
- John Blake, Director for Fair Access and Participation, Office for Students
Session details:
In taking on the role of Director for Fair Access and Participation at the Office for Students, John Blake set out five key priority areas. One of these areas was a need for better evaluation.
Jumping forward to 2024 – there is now a new approach to regulating equality of opportunity which focuses on evaluation being baked into all access and participation work from the get-go.
As the new approach takes effect, where does John feel the most progress on evaluation is still needed? And how can we as a sector work together to plug these gaps and ensure that all students – regardless of their background – can access, succeed in and progress from higher education?
10:30am Busting inequality beyond HE: what actually works?
Speakers:
- Dr Omar Khan, CEO, TASO
- Professor Rachel Brooks, professor of Sociology and Associate Dean Research and Innovation, University of Surrey
- Peter Crowson, Evaluation and Research Coordinator at Nottingham Trent University
- Georgia Roe-Ely, Student at Nottingham Trent University
Session details:
Higher education providers are increasingly held to account for how their graduates fare in the job market but there is little evidence on what works to reduce equality gaps in employment and employability.
This session will bring together leading experts to discuss the current evidence base and what providers can do to make a difference.
11:00am Break
11:30am Robust evaluation: building blocks for success
Speakers:
- Christoph Koerbitz, Chief Research Officer, TASO
- Rain Sherlock, Head of Evaluation, TASO
Session details:
Evaluation is an art, as well as a science. In this session we will navigate the ‘what’, ‘why’ and ‘how’ questions and explore the practicalities of evaluation within the higher education landscape, highlighting the importance of key stakeholders, essential skills, and effective engagement strategies.
1:00pm Lunch
2:00pm Breakout session: Evaluation spotlight sessions – Attainment-raising
Speakers:
- Nicholette Pollard-Odle, Evaluation Officer, TASO
Session details:
Improve your expertise in assessing attainment-raising initiatives through our seminar. Join us for an insightful exploration into the latest attainment-raising initiatives shaping higher education, accompanied by a brief analysis of the evidence supporting attainment-raising efforts. Led by TASO experts, this session offers more than just theory; it provides practical guidance through an interactive exercise. Attendees will gain firsthand experience in evaluating attainment-raising initiatives, empowering them to make informed decisions and drive positive change within their institutions.
2:00pm Breakout session: Evaluation spotlight sessions – Ethnicity degree awarding gap
Speakers:
- Sarah Conner, Research Manager, TASO
Session details:
There is a persistent difference in the proportion of students from marginalised ethnic backgrounds who are awarded a first or upper-second-class undergraduate degree when compared to white students. This is known as the ethnicity degree awarding gap. This seminar session aims to provide evaluation support to those planning and delivering interventions to tackle the awarding gap.
The session will showcase specific examples from TASO’s recent project work with six higher education providers producing theories of change and evaluation plans for their interventions. Practical activities will be embedded throughout and we encourage attendees to come to the session with an intervention or activity in mind.
3:30pm Break – Check-in for overnight guests available
4:00pm New IPE guidance: What works for whom, how and why?
Speakers:
- Dr Emma Vardy, Senior Lecturer (School of Social Sciences), Nottingham Trent University
- Dr Helen Lawson, Research Manager, TASO
Session details:
Introduction to new guidance and tools to support the sector to better understand and conduct implementation and process evaluation (IPE), as part of our recent project with Nottingham Trent University.
4:30pm In conversation: Learning from and influencing senior leadership
Speakers:
- Professor Charlotte Croffie, Pro Vice-Chancellor for Equity, Diversity and Inclusion, Loughborough University
- Jane McNeil, Pro Vice-Chancellor – Education, Nottingham Trent University
- Dr Omar Khan, CEO, TASO
Session details:
Leadership is vital in driving holistic approaches to improving equality of opportunity in higher education.
This session will bring together two Pro Vice-Chancellors to discuss how leadership can move the dial on access and participation work, and the necessary ingredients for success.
5pm – Wrap up and close
6:30pm Drinks reception, followed by dinner
(overnight guests only)
Day 2 programme (8 May)
9:30am Opening and welcome remarks
9:40am In-conversation: supporting disabled students in HE
Speakers:
- Amelia McLoughlan, Network Director, Disabled Students UK
- Dr Omar Khan, CEO, TASO
10:00am Impact evaluation: using quasi-experimental designs in higher education evaluation
Speakers:
- Dr Sonia Ilie, Associate Professor, University of Cambridge
- Mike Kerrigan, Head of Research and Insights (Access and Participation), Nottingham Trent University
Session details:
This session will explore the power of quasi-experimental designs (QEDs) in understanding cause and effect, crucial for informed decision-making and resource allocation. Learn about QEDs, including methods like difference-in-differences and regression discontinuity designs. Delegates will also hear about a case study QED evaluation from colleagues at Nottingham Trent University.
11:00am Break
11:30am Breakout session: Methods made easy – Assessing the quality of evidence
Speakers:
- Tatjana Damjanovic, Research Officer, TASO
- Christoph Koerbitz, Chief Research Officer, TASO
Session details:
A beginner level seminar to introduce the principles of assessing the quality of evidence to develop evidence-informed intervention strategies.
11:30 am Breakout session: Methods made easy – Randomised controlled trials
Speakers:
- Luke Arundel, Research Officer, TASO
- Dr Rob Summers, Research Manager, TASO
Session details:
In this session, we will explore the practical considerations of how to conduct a randomised controlled trial (RCT). Delegates will engage in a live RCT simulation to grasp key concepts of ethical considerations, randomisation methods, sample size determinations, and resource allocation. The session will conclude with an audience Q&A.
1:00pm Lunch
2:00pm Breakout session: Unlocking the evaluation toolbox – Post-entry Mapping Outcomes and Activities Tool
Speakers:
- Dr Rob Summers, Research Manager, TASO
- Luke Arundel, Research Officer, TASO
Session details:
This session introduces the post-entry Mapping Outcomes and Activities Tool (MOAT). The post-entry MOAT is a practical resource designed to enable robust evaluation of student success interventions by assisting practitioners and evaluators to consistently label and record the outcomes and activities they deliver as part of their student success work.
Through a practical activity, delegates will identify target student groups, define outcomes, and classify activities, fostering a deeper understanding of intervention effectiveness.
2:00pm Breakout session: Unlocking the evaluation toolbox – Access and Success Questionnaire
Speakers:
- Dr Kate Jones, Research and Evaluation Manager, The Brilliant Club
- Rain Sherlock, Head of Evaluation, TASO
Session details:
Intermediate outcomes measures are an important tool in our evaluation toolbox. This session will explore the significance of intermediate outcomes and the role of validated scales using the Access and Success Questionnaire (ASQ). Learn practical strategies for impactful evaluation and engage in interactive activities to apply these concepts to your own evaluation.
3:30pm Break
4:00pm Navigating ethics in HE evaluation
Speakers:
- Dr Richard Davies, Higher Education Research and Development Lead, University of Central Lancashire
Session details:
Following recent OfS guidance on publishing Access and Participation Plan (APP) evaluations, when and how to obtain research ethics is front and centre in the minds of the HE sector. This session highlights key considerations for when to obtain ethics approval, and signposts to practical guidance to support the sector as they navigate the ethics processes in local contexts.
4:30pm Close