What we learnt from taking part in TASO’s Summer School Evaluation
Amy Burt, Monitoring & Evaluation Officer, University of Kent
Amy Burt from the University of Kent highlights the opportunities and challenges when taking part in TASO’s Summer School Evaluation.
The University of Kent’s Year 10 ‘Breaking Barriers’ Summer School was involved in the TASO randomised controlled trial (RCT) to evaluate the impact of virtual summer schools. As TASO publishes the interim findings, it provides us with an opportunity to reflect on what we learnt from taking part in this research project in addition to any lessons for the sector.
Developing our outreach curriculum
Being part of the trial gave us the opportunity to speak to groups of both parents and students that we considered to be highly impacted by the pandemic. A number of findings from the Implementation and Process Evaluation (IPE) made us think about how we can adjust our outreach curriculum to accommodate these students (and their influencers), and consider whether they are unique or enduring circumstances.
Despite only being in Year 10 and therefore a few years away from having to make decisions around higher education (HE), the students we spoke to were clear that access to mental health support would be an important factor to them when deciding on university. Perhaps this is due to poor mental health being exacerbated by the pandemic but it’s something that we had already started to consider in activities designed for younger audiences. These findings helped to reinforce that this was the right direction to go in.
Many of our talks designed for parents focus around student finance and funding, usually a major concern for them. Although finances are still a consideration, a big worry for the the parents and guardians that we spoke to was their young person’s grades being a barrier to them accessing and succeeding at university. This may be due to the interrupted education their young person has experienced and something worth exploring in our curriculum design for this audience.
One of the key motivators for a few students taking part in the summer school was just for ‘something to do’; a way to spend their summer –
“Because it’s the Summer and I feel like if I hadn’t have done this, I would’ve wasted my time and not done anything else. So, this feels really productive”
That was something that as an institution we hadn’t considered (or perhaps wanted to consider!) before but we don’t yet know if it’s specific to this cohort – after having 2 years of interrupted study perhaps they’re motivated to do something productive. That’s something that will be interesting to explore this year and going forward.
We would have offered a virtual summer school in 2021 irrespective of our involvement in the TASO research project but it is reassuring that students agreed they had met each of the aims that we had set for the activity and that it did have a positive impact on them. We’ve now been able to adapt this content for in-person delivery this year.
Randomised Control Trials (RCTs) in outreach evaluation
One of our motivations for taking part in this research was because we were interested in the RCT process as a means of being able to demonstrate causality, but reluctant to do it ourselves. Having the support of TASO and the other partners was a real positive influence on our decision to apply. Although the Behavioural Insights Team (BIT) performed the randomisation and selection of participants, the prospect of running an RCT as an evaluator is less daunting than it was before. There are few other outreach programmes in our curriculum that are suitable for this method of evaluation but we will definitely be exploring more Type 3 methods going forward.
The project also allowed us to have additional communication with the control group. Where previously our last communication with unsuccessful applicants was to tell them that they hadn’t got a place, this gave us an opportunity to get in touch with them again and let them know that we wanted to hear from them, and that we valued their thoughts and their opinions. We were able to leave it on a positive note, and we were pleasantly surprised with the response rate.
We look forward to the final research findings once we know whether these students applied to and enrolled in Higher Education or not and can see any differences between the control and treatment groups and any impact that may summer schools had.
Amy Burt, Monitoring & Evaluation Officer, University of Kent