Vol. 21.1
The challenge of quality: editorial
The challenge of quality
David Law, Editor
Perspectives
Does it ever seem to you that the more we talk about something the less we know about it?
Perhaps the evaluation of quality in higher education (HE) is one of those slippery subjects that we can never manage to pin down. When I was first employed by a university,
in a seamless transition from postgraduate student to lecturer, nobody (apart perhaps from my Head of Department) seemed to mind very much what I taught or how I taught it. The key issue was whether students could make it into final examinations and how they performed once there. In a radically different HE system, set up to serve a small minority, the filter was A-level grades and the quality controls were about inputs and outputs. The rather important middle piece, the actual education that took place, was very much unsupervised by the institution. I do not think it occurred to many of us to even ask if students were satisfied. ‘They’ were in lectures and tutorials to learn, and ‘we’ were there to teach.
Now it has all changed, although this has taken at least a generation. As this issue goes to press, the teaching evaluation framework (TEF) is being built in offices (and perhaps even workshops) somewhere near Bristol. Readers, no doubt, will already know that the TEF will consider universities and colleges based on metrics (e.g. graduate employment, student retention, and student satisfaction) and additional evidence submitted by the institutions under scrutiny.
In May 2017 evaluations from stage two of the TEF will be announced. That will be a trial year; all institutions meeting basic standards will be allowed to raise fees in line with inflation (and surely all providers meet basic standards). Differential fee increases may be introduced the following year. But even before then, the real impact on HE will be that prospective students will be selecting where they choose to pay large fees by reference to judgements that are to be communicated using the codes: ‘bronze’, ‘silver’, and ‘gold’.
Be careful what you wish for: after consultation, the proposed ratings (originally ‘meets expectations’, ‘excellent’, and ‘outstanding’) have been changed, in an Olympic year, to the colours of medals available in competitive sport. Is this what HE has now become?
To access the full version of Perspectives vol. 21, please visit the Taylor and Francis website.
It does sometimes seem that highly intelligent individuals, working together on problems of common interest, end up making strange decisions. It could not have been difficult to see that there is not much difference between ‘excellent’ and ‘outstanding’, as the consultation concluded. How long will it be before the penny drops and there is an acceptance that awarding ‘medals’ for performance still requires definition of terms and the descriptive use of words? In a race, it is simple to decide first, second, and third places but higher education institutions (HEIs) are not running in the same race. Ultimately, unless we are prepared to accept the kind of regulation that has always been ruled out, an institution defines its own purposes and quality is, therefore, a relativistic concept.
Of course this is recognised, at least in theory. The last consultation paper included the general principles that underpin TEF. There are some fine turns of phrase here about bureaucracy, diversity, autonomy, transparency, and flexibility. They sit alongside a potentially mechanistic process where calculation is likely to trump peer assessment.
When we announced that the AUA John Smiths Essay competition in 2016 would be for work that considered the ‘Challenge of Quality’, we thought we might have some submissions that tried to explain what the emerging new system is all about. It seems that we must wait, perhaps until we have some results from the TEF. However, as your editor, I am pleased to say that we had some excellent essays to choose from this year.
The winning piece was written by Namrata Rao and Anesa Hosein, both of whom are academic colleagues, working on HE, at English universities. The research that their essay is based on was supported by the Quality Assurance Agency (QAA). It has also resulted in another piece that surveyed the websites of 38 HE institutions and found a significant gap between the expectations of government and the provision of learning and teaching (L&T) information on websites. Their winning essay uses a small-scale survey of professional staff; Rao and Hosein analyse data from interviews to identify the reasons for the information gap. Their intention was to reveal the ‘core issues which hinder the provision of quality L&T information’.
To access the full version of Perspectives vol. 21, please visit the Taylor and Francis website.
The full survey of websites found only ‘limited adherence’ to the expectations communicated to the sector by QAA guidance. Although the researchers found ‘a consistently large amount of information on student workload and on methods of learning, teaching, and assessment’, the information provided often lacked specificity. Only one-third of websites provided ‘expected contact time with tutors’ and only one-fifth went into any detail about the balance between seminars, lectures, and (where appropriate) laboratory sessions.
The limited scale of the survey, in the words of the authors, ‘does not permit unequivocal conclusions’. It is unfortunate that only a small minority of institutions chose to participate: for reasons that are explained in the essay, when 30 of the original 38 institutions were contacted only 8 found themselves able to agree. More than half either did not reply at all, or failed to respond after making initial enquiries about the project. This must raise questions about how solid is the sector’s commitment to the new approach that underpins TEF. Overall, Rao and Hosein state: ‘our findings suggest that many HEIs adopt a procedural and formulaic approach to information provision’.
The prize jury also shortlisted two other essays that appear in this issue and decided to make an award to one of them. ‘New developments in transnational education and the challenges for higher education professional staff’, by Michelle Henderson, Rebecca Barnett, and Heather Barrett, was highly commended. The new developments that they call attention to fall into two broad, possibly overlapping, categories. ‘New technologies … have given rise to new partnership types and models … [creating] a market for content providers … [and] the emergence of new education providers such as publishers, content aggregators … and professional bodies’. Secondly, there are ‘new partnership arrangements that involve multiple agencies and third parties’ often with a ‘broker’ or ‘intermediary’ at the centre of a web of complex relations. ‘Increasingly, the broker’s role is to provide advice and expertise to both sender and host HEIs’. As the authors of this essay note there are ‘ever more complex challenges and risks associated with quality assurance, academic standards, and student experience’. The concluding paragraph of this essay should be taken very seriously by all those involved in international delivery of HE. Although some might be cautious about revealing ‘trade secrets’, fearing loss of competitive advantage, those who care about the strength of the ‘Education UK’ brand do need to appreciate that there is ‘much to be gained from extending and widening the dialogue between and across professional service staff in different organisations’. We hope that our own organisation, the Association of University Administrators, will pay careful attention and use its own International HE Network to encourage such a dialogue.
To access the full version of Perspectives vol. 21, please visit the Taylor and Francis website.
Our third article, ‘Collaborative provision quality assurance isn’t just red tape … ’, by Claire Hughes and Helen Thomas of Southampton Solent University, reports on an institutional project that originated in the creation of a specific department to support academic partnerships. The authors chose 12 individuals, including academics, colleagues in the professional services, and external stakeholders, and invited them to participate in semi-structured interviews. All respondents had experience in the development of academic partnerships.
Claire Hughes and Helen Thomas undertook this project as a way of consolidating the ‘quality culture’ of their institution, and to streamline development processes and innovation. To implement ‘refreshed business processes’ to support collaborative provision ‘in an inclusive way’, they involved colleagues and peers in ‘the development of staged guidance materials aimed at promoting engagement … [and] fostering the growth of partnerships to their full potential’. This kind of practitioner research is of great interest to our journal and we invite others who have undertaken such projects to write them up for publication.
Finally, we have two more pieces that came to the journal from authors attracted by the thematic issue (rather than the prize fund kindly provided by the John Smith Group). For some authors, and for some subjects, inclusion in a themed issue is the ‘reward’.
‘Student evaluations of teaching: improving teaching quality in higher education’ comes to us from four authors in the Division of Psychology at Troy University, USA. This article discusses research on the administration and interpretation of student evaluations of teaching (SET) in both USA and UK. It considers various problems associated with SETs, including the key issue of how to engage students. Given the large investment in SETs in many universities it is clearly important to maximise the practical information gained.
Frank Hammonds, Gina Mariano, Gracie Ammons, and Sheridan Chambers conclude that ‘SETs, while they do have shortcomings, provide valuable information regarding teaching effectiveness’. SETs have become a popular, relatively efficient means of obtaining feedback on instruction in HE in the USA, the UK, and in many other parts of the world.
To conclude, ‘Revising the Research Excellence Framework: ensuring quality in REF2021, or new challenges ahead?’ changes the focus away from teaching to research. Tony Murphy, an academic colleague at Sheffield Hallam University, considers the likely shape of the next UK Research Excellence Framework. The paper explores some of the recommendations from the 2016 Stern Review. The issues of ‘burden’ and ‘gaming’ are explored.
The editorial team at Perspectives has enjoyed the challenge of putting this issue together. This editorial is being written only three months after the closing date for the 2016 AUA John Smiths Essay Prize. We very much value the co-operation of our publisher, our reviewers and, above all, our authors. Without this, ‘The Challenge of Quality’ would have been even more of a challenge.
To access the full version of Perspectives vol. 21, please visit the Taylor and Francis website.