Strategic use of management information to monitor how effectively your university is improving graduate employment outcomes.
or
Counting chickens before the horse bolts.

Given the impact of the Teaching Excellence and Student Outcomes Framework (TEF), and the Office for Students’ (OfS) requirements for registration, graduate employability, is now more significant than ever.

Kate Dodd
Consultant in higher education

The extent to which a higher education providers’ student destination results meet, or exceed, what the OfS considers an ‘acceptable baseline’, and evidence that it is striving to deliver successful outcomes for all its students – regardless of their background – are central to the current regulatory and quality assurance regimes. However, the measures most universities rely on to monitor the effectiveness and reach of their employability support, leaves much to be desired. The change from DLHE to the Graduate Outcomes survey leaves university managers with an arguably less clear, and definitely less current picture of how well their enrolled students are being prepared for successful early career outcomes.

While the move from the provider-managed DLHE survey to the HESA-managed Graduate Outcomes survey has addressed the long standing allegation that many universities were gaming the system in their data collection; the reduced return rate and the increased delay between graduation and the census date mean that the results of Graduate Outcomes will be even less satisfactory as a measure of the performance of the institution’s careers and employability provision for current students.

Typically, when senior managers receive reports on their careers and employability provision these focus on the following;

• results of the latest cohort of destinations – and variance from previous cohorts,
• the satisfaction of those students who have used the services,
• the range of and feedback from employers with which the services engage.

There may also be some reporting of new initiatives that the service hopes will address areas of weaker employability.

Many providers survey each cohort of students’ career readiness at enrolment each year. Where this is done some valuable metrics and targets can be used to monitor and improve the extent to which students are preparing for their future careers. The Careers Group London has undertaken a Learning Gain Pilot project on the use of this careers registration data and how it could measure students’ readiness for work and predict their future employment outcomes.

What gets missed from these reports is an analysis of the extent to which students are accessing the services, and whether services are reaching those students who are most at risk of ending up in outcomes that the TEF and OfS consider to be less than positive. The needs of these students are probably what the OfS has in mind when it refers to the existence of significant differences in outcomes between students from different backgrounds as an indicator of poor performance against condition of registration B3.

Many of the larger providers use software that provides reports on the proportion of enrolled students who leave a digital footprint in the systems for accessing careers advice information and guidance and for attending careers events. University-managed careers management systems – such as TARGETconnect and CareerHub – can provide reports on users and, significantly, non-users of the careers and employability provision. They can do this because their user population is taken from the student record and not just comprised of those students who choose to opt into using the on-line provision. Smart managers are monitoring the engagement of enrolled students to target initiatives and marketing where it is most needed in-year and to minimize the numbers of students who fail to benefit from the employability support that is provided.

Visiting a number of providers which use such systems recently I was struck by the extent to which data analysis skills in the careers team presents a barrier to making effective use of this valuable business intelligence. Some also reported difficulties accessing support from colleagues, such as in the Planning Office, due to this data not contributing directly to any compliance or reporting requirement. This CPD or skills gap in the careers team is something which senior mangers overlook to their cost. Relying on graduate outcomes as the primary performance measure means that the university is only ever evaluating the effectiveness and reach of their careers provision after the graduate ‘horse has bolted’.

Most university senior management teams review admissions data very regularly to monitor how well they are performing (in-year) in attracting applicants, and to forecast future admissions. Evidence of university senior teams taking a similar interest in how well the institution is preparing students to become successful alumni by optimizing their career potential is hard to find.

AGCAS, the professional body for careers practitioners in UK higher education, has recently launched a working party looking at the use of data insights into ‘on-course’ students . It will consider, explore and feedback to AGCAS members on;

• the implications of student characteristics – such as commuter students,
• the use of careers registration data which illuminates career readiness, and
• on postgraduate research data.
• It will also explore ‘impact measures’ for careers and employability work.

I hope that a measure of the extent to which services and interventions are accessed by enrolled students will feature highly in this group’s work. It would benefit from strong links with colleagues in the AUA, Higher Education Strategic Planners Association (HESPA) and Academic Registrar’s Council (ARC) who are also working on data analytics and performance indicators.