Case Study 2: Cluster – Randomised Control Trial (RCT)
A cluster randomised control trial to evaluate the effectiveness and cost-effectiveness of higher education/career interventions with Level 3 students from low Index of Multiple Deprivation (IMD) areas
Project Description and Aim:
Heads of year in Further Education (FE) Colleges have a key role in guiding students and, as research has indicated, a strong commitment to raising aspirations. It was unclear whether an intervention based on Cognitive Behavioural Therapy (CBT) principles led by these teachers could improve the likelihood of their students applying for an HE course. Thus, this study aimed to evaluate the effectiveness and cost-effectiveness of a targeted advice intervention provided by FE heads of year in the nine months before university applications were made.
This study was designed as a cluster randomised, multi-centre, controlled trial. The participants were students aged 16 and older, taking courses that could lead to HE and in the lower two quintile areas according to the IMD. The study was conducted in 15 local authority areas of the UK between March 2014 and December 2015. After stratification by local authority, FE colleges were randomly allocated to group A (training and delivery of the intervention at the beginning of the study) or B (training and delivery 9 months later, i.e. to the next student cohort). The allocation used computerised random number generation in blocks of 5. Each FE college recruited five students, with both groups followed up for 9 months.
Methods:
The intervention consisted of four systematic, structured, face-to-face consultations with the head of year, covering the student’s assessment scores and aspirations, the process of applying for an HE place, including how to choose a course, possible career possibilities and outcomes for the student. It also provided an opportunity to discuss possible barriers to the student applying for an HE course. The consultation plans were informed by CBT but were not therapeutic in style or content.
The primary outcomes were self-reported beliefs about the likelihood of applying to HE (assessed at 6 months using the 12-item TICCCL) (Trowsdale Indices of Confidence in Competence, Creativity and Learning) vs actual application to HE. Secondary outcomes were strong future career narratives, an understanding of possible future educational paths, and the cost-effectiveness of the intervention compared with usual advice. Although blinding was not possible for either teachers or students, the assessment of outcomes was conducted by researchers blind to group allocation.
Key ethical considerations:
Three key ethical issues were identified in this study: informing students and FE college staff and gaining their consent, addressing the unequal treatment of intervention and control groups, and the possible harm caused by over-raising students’ aspirations.
The project team worked hard to develop a series of information sheets and a video for students and FE college staff. The project coordinator also had online conversations with key senior members of the FE colleges involved, including the head of year, covering the nature of the project in more detail and the inclusion criteria for selecting students. The team also provided support documents for teachers to use in introducing and discussing the project with potential student participants. All student-facing material was piloted for intelligibility for 16-year-old students.
FE college leaders were asked for consent to conduct the study in their college. Heads of year and students were asked for their consent to be involved. As all students were over 16 years of age, they were deemed competent to give their consent, and parental consent was not sought. However, a short information sheet was given to student participants to hand to their parents. This invited parents to contact the college or the research team if they had any concerns.
While the key purpose of the study was to find out whether the intervention had any impact on students’ applications for HE courses, there was anecdotal/intuitive belief amongst FE staff that the intervention would be beneficial to their Continuing Professional Development and Learning (CPDL). The design was, therefore, costed to include CPDL and support materials for the control group FE colleges to use in the following year. Given that the intervention had implications in terms of demanding time from students and FE college staff, there was no clear evidence that it would be beneficial and, after ethical scrutiny, it was agreed that a principle of equipoise existed between the two arms of the randomised control trial (RCT).
Finally, there was a risk that engaging in this additional intervention might unrealistically raise the aspirations and expectations of the students in the intervention group. This was discussed with FE staff in the preliminary conversation about inclusion criteria. The participating students needed to be capable of successfully applying to an HE course, and the intervention conversations were to be challenging but realistic in their evaluation of career/educational aspirations and goals.
Scientific limitations and recommendations for future research:
It is essential to identify and avoid bias in research, but issues with sources of bias may have occurred in this research. For example, the teachers providing the intervention were responsible both for selecting students and entering the data collected onto the electronic template; both tasks are possible sources of bias. The overall population included more males than females (p < 0.01).
However, none of these issues was deemed sufficient to conclude that the study would not provide valuable scientific evidence about career/HE interventions.