New foundations for understanding the impact of HE outreach

TASO’s goal is to generate more robust, or causal evidence, on the impact of higher education activities designed to promote equality. To achieve this, we must first establish a solid foundation of existing evidence to build upon

Today, in partnership with the Higher Education Access Tracker (HEAT) Service, we launch a new analysis of their extensive data – covering over 100,000 students – to provide a better understanding of the relationship between participation in outreach activities, attainment and progression to higher education. Because we can only look at correlations, or existing patterns in the data, the analysis does not tell us whether certain outreach activities cause students to do better at school. Or if the activities directly influence decisions to enrol in higher education. But it does provide some very important new insights to help the sector. An investigation into the relationship between outreach participation and KS4 attainment/HE progression finds that taking part in more intensive outreach is associated with higher Key Stage 4 (KS4) attainment and higher HE progression.[1] Why is this important? Well, national research has shown KS4 attainment to be critical to future progression to higher education and it also opens doors to a whole range of future opportunities to students.

Laying the foundations: which approaches look promising?

We analysed the relationship between summer schools, campus visits and mentoring and KS4 attainment / HE progression and here’s what we found.

Summer schools

Participation in summer schools is associated with higher KS4 attainment and higher HE progression.
Taking part in a summer school is associated with an increase in Attainment 8 scores of 2.9 points.[2]
We estimate that HE progression is 5-14pp higher for those who attended any summer school in our data versus those who did not in. We also estimate that top-third progression is 3-12pp higher for those who attended any summer school versus those who did not.

Campus visits

Participation in campus visits is associated with higher attainment at KS4, albeit less so than participation in summer schools.
Taking part in a campus visit is associated with a small increase in Attainment 8 scores of 0.7 points. Differences were greatest for disadvantaged students with low prior attainment.

Mentoring

Our analysis of mentoring produces mixed results on its association with KS4 attainment and there is a negative association between HE progression outcomes and mentoring.
One possible reason for this result is that mentoring participants are specifically chosen because they face particular challenges and barriers in education.
Therefore, the association between mentoring and less positive outcomes could be due to the type of individuals who take part in these activities, rather than the activities themselves.

The analysis reveals initial positive support for summer schools – which is backed up by existing evidence on positive associations relating to higher education participation. However, it does not account for other factors such as an individual’s motivations. In contrast, the findings on mentoring somewhat challenge the existing evidence that shows positive associations between mentoring and attitudes / aspirations and, in some cases, attainment and HE progression. Why might this discrepancy exist? While we can’t say for certain, it could be that there are a big differences in the design and implementation of mentoring programmes across the sector. Our analysis aggregated all programmes in the HEAT dataset, mixing this diverse range and quality of programmes together in our findings. And importantly, we need to remember that this analysis is correlational – so it’s possible that our findings are due to existing differences between the individuals who take part in the different types of activities. In other words, it may be that students with lower grades who are less likely to go to HE are more likely to take part in mentoring, rather than the other way around – we just can’t tell in the data.

A partial view: more robust evidence needed

The foundational insights in this analysis do not provide us with enough information to fully understand all the factors at play. For example, the data does not tell us about the role of parental support or individual motivation, which we know are strongly associated with attainment and HE progression. This means that there is a risk of inherent bias in this sort of analysis, because the groups we examine (i.e., those who do and do not take part in outreach, or in certain activities) may have been very different to begin with, regardless of those activities.[3] We now need to build upon these new insights to develop more robust causal evidence, so we can be certain it is the activities that are causing the difference in outcomes. At TASO we working on several research projects to do just this. We are running a randomised controlled trial (RCT) of summer schools to uncover the effect of the activity versus the effect of certain individuals being more likely to attend. Working with eight universities to implement a collaborative RCT, we will randomly allocate eligible individuals to fill the available places on the programme, or not, and then the outcomes of both groups tracked over time to measure the impact of the activity. On multi-intervention outreach and mentoring, we are working with Aston University, Kings College London, and the University of Birmingham to develop a more thorough understanding of the different elements of mentoring activities used in outreach programmes. We hope to produce a common recording framework which providers can use to capture the different facets of the mentoring they deliver. Agreeing upon a more consistent approach across the sector will help future research to delve into the most effective elements in more detail and lay the foundations for more nuanced analysis on the impact of this activity in future.

Download the summary report: An investigation into the relationship between outreach participation and KS4 attainment/ HE progression
Download HEAT technical appendix 1
Download TASO technical appendix 2

[1] An ‘intensive’ package of activities is defined by HEAT as: one or more summer schools; one or more HE insight events; one or more mentoring interactions; one or more projects; two or more skills and attainment activities; two or more campus visits; one or more skills and attainment activities and one or more campus visits; three or more HE information talks and one or more skills and attainment activities; three or more HE information talks and one or more visits. [2] For more information on Attainment 8 scores, please see this overview from the Department for Education [3] For more information on selection bias please see this explanation on the Institute for Work and Health website.

Navigation breadcrumbs