A new initiative to measure the research output of Trinityâs staff, currently in its pilot phase, has raised concerns that it might potentially rank and compare individual academics, already drawing criticism from a trade union.
The new initiative, Principal Investigator Quantitative Analytics, is proposing to rank academics based on their research output. The initiative has faced criticism in College, including from the Irish Federation of University Teachers (IFUT).
Speaking to The University Times in his capacity of Trinity Branch Chair of IFUT, Dr John Walsh of the School of Education said that âI think there are concerns among many staff members, and certainly IFUT members, throughout College. We think that itâs really counter to the whole spirit of the academic communityâ.
âMuch of PIQA [Principal Investigator Quantitative Analytics] is quite reasonable and consists mainly of information gathering about, say, research income and other aspects of research, which we would have no objection toâ, Walsh added.
However, describing the proposed idea of ranking academics as âproblematicâ, Walsh commented that it is âeffectively individual benchmarking or individual league tablesâ and that it would âpotentially involve comparing individual academics based on a small number of criteria such as grant applications, total journal articles or web of science citationsâ.
A response issued by College spokesperson, Caoimhe NĂ Lochlainn, on behalf of the Dean of Research, Prof John Boland, confirmed that the project was in its pilot stage, but declined to comment until the initiative had been fully rolled out.
In the UK, the Research Excellence Framework is used to assess the quality of academic research, with results used to determine how much research funding universities and academics are granted, as well as an institutionâs rankings in league tables. Under the framework, 20 per cent of the overall score is for research outside of academia. The framework has been criticised for discouraging long-term research projects. The framework has also been criticised for adding layers of bureaucracy and expense, with Times Higher Education reporting that some universities appeared to be “gaming” the system.
Walsh states that IFUT would “object to anything that involves an approach” similar to that of the Research Excellence Framework: “The English system of measuring research outputs and funding based on research outputs is toxic and has been incredibly damaging to English universities and is under a lot of criticism in England.”
“While it’s not a fully fledged product of the English system”, Walsh continued, “it does bear some of the hallmarks of the English system, and certainly IFUT is entirely opposed to anything that would resemble [it]â.
“In fairness, it’s not clear that that’s the intention of the authors of this, but there are major problems. This would be the first step down the road towards an [English] style system, if adopted.”
At a Research Committee meeting in November 2016, Prof Trevor Spratt, Director of Research for the School of Social Work and Social Policy, agreed that âthe tool was valuableâ but warned against its use in the future, commenting that âsuch metrics were used in the UK in a very hard line way when assessing staffâ, and said that he âwould like to see reassurances on the future use of such a toolâ.
Also raising a similar concern at the meeting was Prof Peter Stone, who was filling in for the School of Social Science and Philosophyâs Director of Research, over the potential future use of the metric and the dangers of following a system similar to the UKâs Research Excellence Framework.
Outlining aspects âthat are very worryingâ, Walsh spoke of two elements, the first being that such ranking of the academic community might be done in a way âthat encourages internal competition between academics within a school or within a disciplineâ.
At the same meeting of the Research Committee, Prof MicheaĚl OĚ SiochruĚ, then-Director of Research for the School of Histories and Humanities, who has since been appointed head of the school, stated that his main concern was also the motivation behind the generation of a tool that may divide the College and âwork against the spirit of collegialityâ.
Walsh did, however, acknowledge the fact that the Dean of Research, John Boland, who is heading up the project, âhas been open to discussing it at various foraâ.
The second element raised by Walsh is the fact that the College has, over a number of years, âand more recently the arts and humanities faculty, developed really inclusive research metrics that recognise everything from a standard monograph or a peer-reviewed journal article to a play or editing a manuscript as valid forms of research outputâ.
Though extremely supportive of the Research Officeâs work in improving Trinityâs research profile, Walsh said that âthere will be no point in undermining the faculty’s own, and the College’s own, inclusive research metrics, by introducing something that was much more fine grained, was much more limited and which would encourage competition among academics that is really counterproductiveâ.
âI think a key issue here is that there is a tendency in academic institutions to introduce flawed systems of metricsâ, Walsh added.
On the question of the anonymity of the data, Walsh said: âI think whether itâs anonymised or not, it promotes a kind of internal competition and benchmarking that’s very damaging to collegiality and it’s another set of metrics on top of research metrics that we’ve already recently agreed.â
âI think anonymising it may be designed to offer assurance, but I think it misses the pointâ, Walsh added. âThe concerns that people have are much more fundamental than whether it’s anonymised or not. I don’t think it would be possible to relieve people’s concerns if you could find a way to anonymise it perfectly.â
âI don’t want to be competing with my colleagues, I want to be working with them for the benefit of the institution. I think that’s what we should all be aiming forâ, Walsh added.
When asked about the fact that the current metrics of the initiative do not account for those who take maternity leave, Walsh said that that was a âkey concernâ, describing the proposed metrics as âa blunt instrument that would be crude and unfair, both to colleagues carrying extra administrative burdens, but also to those coming back say from maternity leave or other forms of leaveâ.
Walsh explained that the metrics also do not take into account the fact that you may be a College Tutor, or a director of undergraduate teaching and learning, or a programme coordinator, or other such activities which may make an individualâs research be âover a couple of years less than it would otherwise beâ.
Echoing this concern at the meeting of the Research Committee, in November, Prof Helen Sheridan, Director of Research in the School of Pharmacy and Pharmaceutical Sciences, raised the point that the âmajority of academics were balancing curriculum delivery and development, administration and research, and there are no balances, measures or tools to measure productivity across the spectrumâ.
When asked about the motivations behind the initiative, Walsh said: âThe motivations to me are very obscure. I can see the motivations behind PIQA [Principal Investigator Quantitative Analytics], the overall project, and that’s to get as much information as possible and to enhance the College’s research profile.â
âIn fact, you could say that all but one of the elements of PIQA [Principal Investigator Quantitative Analytics] are broadly quite reasonable, but one element, which is individual benchmarkingâ, Walsh stated.
âWe’re benchmarked through the promotions process, and that’s an extremely rigorous processâ, he added. âWe’re more generally benchmarked through the research metrics so it just seems pointless to invent another system of metrics.â
Walsh emphasised that the initiative is still in early stages and would need to be brought to University Council through the Research Committee. âI think it’s very important that before something as controversial and as flawed as this policy is brought forward for consideration, there needs to be very full consultation within the college communityâ, he stated, adding that âthere are signs that that has been understoodâ.