The road less travelled: evidence of effective interventions for care experienced and estranged students

The Unite Foundation is a charity with a 10 year history supporting care experienced and estranged students in partnership with higher education providers (HEPs).  In this blog we speak to them about their journey toward more rigorous impact evaluation, the considerations of working with small populations and their reflections on working with complex programmes.

1. What were the main findings from your recent evaluation? 

The Unite Foundation gives free, year-round, accommodation to care experienced and estranged students to ensure that we can narrow the current gap in educational outcomes. Students have told us that the scholarship makes a difference but, previously, we had no way of evidencing the impact.

Independent analysis, carried out by JISC, examined how Unite Foundation scholarship students compared to their peers focusing on four metrics:

• progressing beyond their first year;
• moving from years 2 to 3;
• completing their degree; and
• achieving a ‘good degree’ – either a 1st or 2:1.

The report found that Unite Foundation scholarship students outperformed other care leavers in every metric. The most striking finding was that scholarship students progressed from year one to year two on a par with non-care leavers.  Three comparator groups meant we could look at care leavers at the same partner institutions, care leavers from outside our own network of HEPs, and non-care leavers. We also controlled for known influencing factors such as  gender and prior attainment.

2. What were the challenges you faced when conducting this evaluation?

Even with 10 years of scholarship student data, there were still challenges based on sample size; a perennial issue with minority populations.  That, combined with tracking completion and grade outcomes with long lead times, meant for those metrics to be robust:

• we could not include hundreds of scholarship students currently studying
• we restricted our analysis to a very stringent course completion period of three years. This meant we could not capture  students completing longer course pathways (including Scottish, sandwich courses or those that re-sat a year for example)
• not including the longer pathway students meant our sample was large enough to control for some factors as mentioned above, but not for examining some other factors (for example disability)

Despite these challenges, we’re pleased to  have reached the TASO and OfS’ Type 2 standard of evidence.  But there is more work to do to reach Type 3 causal evidence.  To achieve that, we’re exploring both a greater sample size (powered ‘n’) and  looking at small ‘n’ impact evaluation methodologies.  On either avenue, we’re really keen to collaborate with others on the same journey to help us all reach that goal sooner.

3. What do you think we need to know about care experienced learners that existing research can’t tell us? 

As a sector, to drive change we need to be able to regularly access data on these populations.  At a national level, additional work is underway to reflect estrangement and levels of care experience within the Higher Education Statistics Agency (HESA) and the Universities and Colleges Admissions Service (UCAS) frameworks, which is especially welcome. It would assist evaluation efforts if this data national-level data was then publicly available on an annual basis.

Similarly, it would aid evaluation efforts if existing national insight pieces like the National Student Survey (NSS) and the HEPI/Advance HE Student Experience Survey contained flags for care experience and estrangement so that the sector more widely can understand this section of the student population.

If senior leaders within HEPs cannot see the data about estranged and care experience performance within their institutions, they are not able to monitor the commitments made in Access and Participation Plans (APPs).

4 . How do you think we could improve the evaluation of interventions that aim to support care experienced learners? What can you do? 

1.  Follow us throughout the Autumn term to catch our open call for more HE partners in the scholarship scheme.  This is our core effort to grow the scholarship population and be able to test for Type 3 causality.

2. If others in the sector are evaluating interventions with care experienced and/or estranged students at an institutional level, please get in touch so that we can come together and grow a special interest group.

3. Express interest in developing and implementing an evaluation ‘capsule’ – consistent measures of housing fragility, disrupted learning and adverse childhood experience – that can be used by all of us across any of our interventions with these students.  In this way we can grow a body of evidence as a sector that none of us could achieve alone.

Find out more about The Unite Foundation here.

Read TASO’s report on supporting learners with experience of children’s social care here.