Blog21 October 2025
Can a pre-university online course lead to better pass rates?
Professor Billy Wong, Director of Research and Evaluation, and Lydia Fletcher, Research and Evaluation Manager, University of Reading.
Providing students with access to an online transition course before starting university may be a contributing factor in providing a ‘modest but meaningful’ positive impact on success in students’ first year of university.
Universities typically provide a variety of activities and resources – from targeted transition events to formal induction programmes – to help students adjust in their transition to university. Yet the success of this transition can depend on students’ prior experiences and expectations, and how much they engage with the suggested transition support.
Tracking data on engagement with transition activities, and linking this data with the rates of passing Year 1 can help to establish the success of such activities for student continuation.
Using quasi-experimental methods
To consider the efficacy of transition activities, we decided to evaluate the data related to a specific intervention at the University of Reading: a pre-entry online course, which aims to prepare students for university life. Using a university-wide online transition course as our example, and rates of passing Year 1 as our proxy measure of success, we analysed institutional data from more than 28,000 students across seven academic years (2016/17 to 2022/23).
This is described fully in our paper, Assessing the impact of a university transition online course on student continuation using statistical matching methods. The paper demonstrates how doubly robust estimation – which combines propensity score matching and outcome regression – can be used as an example of a quasi-experimental design. This approach can offer robust evidence of impact for higher education evaluation.
The transition course and our evaluation question
The transition course aimed to equip and prepare students with academic study skills prior to their arrival. The course was delivered online to make it easier to access. Our evaluation question was whether enrolment in this optional online transition course impacted students’ outcomes, specifically passing Year 1 or not at the end of the academic year.
Institutional data as a resource
Our analysis drew on existing student records collected by universities for statutory reporting, including information on sex, domicile, age, ethnicity, disability, household income and POLAR quintiles. As such, this data is available across the sector, which makes the analysis replicable and reduces the need for additional data collection.
A difference in pass rates
The descriptive analysis found a consistent gap in pass rates of around 7 percentage points between 2017/18 and 2022/23, with non-enrolled students passing at 83.2% and enrolled students at 90.2%. However, these comparisons do not control for differences in student characteristics.
To control for those differences, we used statistical matching methods. We used matching (propensity score matching) followed by regression on the matched sample (doubly robust estimation). The aim here is to create a comparison group with similar characteristics between those who did and did not enrol in the course. We used existing markers already collected as part of widening participation reporting, such as ethnicity, domicile, disability and socioeconomic status.
We could not include UCAS entry tariff points (a measure of prior attainment) in the main model, as the dataset was incomplete, particularly for international students. In a subset analysis using available tariff data, we did find a similar gap, though we recognise this was only a limited test. We then used regression on the matched enrolled and non-enrolled groups to arrive at our estimate based on doubly robust estimation.
A modest but meaningful impact
The doubly robust estimation results showed that students who enrolled were 6.3 percentage points more likely to pass Year 1 than those who did not enrol (according to the average treatment effect, a measure used to estimate the causal effect of a treatment or intervention on an outcome). This is very similar to our descriptive analysis but we can be more confident that this figure was calculated based on a matched sample, which controlled for observable differences that we have data on.
Based on the university’s student population, we estimate that around 100 more students per academic year would have passed Year 1 if they had enrolled in the online transition course. While this is an estimate rather than a guarantee, the potential gain is institutionally meaningful with better student continuation data and income from tuition fees.
We know there are unaccounted and uncontrolled factors in these calculations, such as, in our case, data on student motivation and engagement. As such, the 6.3 percentage points difference should be read as modest but meaningful, as one contributing factor among many that influence student success.
Statistical matching methods alone do not explain why the course worked or how it was experienced by students. It is important that different methods and data are gathered to provide a more holistic account of the effectiveness and impact of these interventions and activities.
While doubly robust estimation can help to safeguard against model misspecification (where the model might not reflect reality, for various reasons), it does not remove the risk of unobserved factors confounding the results, so future work should continue to test and refine these approaches.
Why this matters for evaluation practice
This matters, as there is a wealth of – currently underutilised – data available in large institutional datasets, that could be used to drive interventions and improve student outcomes. It is also important to note that this work contributes to the relative lack of evaluation studies on university-wide educational interventions that use advanced statistical matching approaches, as noted in this TASO project.
This worked example, including challenges and caveats, shows how evaluators can use existing student data to explore the type of causal evidence that is required to design effective interventions and improve outcomes for students.
- View the open-access paper in Evaluation Review.