论文标题

大型团队研究是否在国家研究评估中?英国研究卓越框架的案例2021

Is big team research fair in national research assessments? The case of the UK Research Excellence Framework 2021

论文作者

Thelwall, Mike, Kousha, Kayvan, Abdoli, Mahshid, Stuart, Emma, Makita, Meiko, Wilson, Paul, Levitt, Jonathan

论文摘要

协作研究引起了研究评估问题,因为很难公平地归功于作者。在其作者之间将文章的奖励分开,表面层面是最大的公平性,但无论团队规模如何,许多重要的评估为每个作者提供了全部信誉。这样做的基本原理是减少劳动力,以及需要激励协作工作的必要性,因为有必要解决许多重要的社会问题。本文评估了在英国研究卓越框架(参考)2021的情况下,与分数计数相比,是否进行了全面计数的变化结果。对于此评估,根据评估单位(UOA),分数计数将期刊文章数量减少到全部计数值的10%。尽管有很大的差异,但基于全面计数或分数计数的总成绩平均值(GPA)的结果为结果中位数为0.98以内的Pearson相关性提供了结果。最大的变化是考古学(r = 0.84)和物理学(r = 0.88)。较高的评分机构因分数计数而损失较高的趋势,在34个UOA中,损失具有统计学意义。因此,尽管从公平的角度来看,对协作撰写的产出的贡献明显过度加权似乎并不是太成问题了,但在少数UOA中,它带来最大的差异可能值得研究。

Collaborative research causes problems for research assessments because of the difficulty in fairly crediting its authors. Whilst splitting the rewards for an article amongst its authors has the greatest surface-level fairness, many important evaluations assign full credit to each author, irrespective of team size. The underlying rationales for this are labour reduction and the need to incentivise collaborative work because it is necessary to solve many important societal problems. This article assesses whether full counting changes results compared to fractional counting in the case of the UK's Research Excellence Framework (REF) 2021. For this assessment, fractional counting reduces the number of journal articles to as little as 10% of the full counting value, depending on the Unit of Assessment (UoA). Despite this large difference, allocating an overall grade point average (GPA) based on full counting or fractional counting give results with a median Pearson correlation within UoAs of 0.98. The largest changes are for Archaeology (r=0.84) and Physics (r=0.88). There is a weak tendency for higher scoring institutions to lose from fractional counting, with the loss being statistically significant in 5 of the 34 UoAs. Thus, whilst the apparent over-weighting of contributions to collaboratively authored outputs does not seem too problematic from a fairness perspective overall, it may be worth examining in the few UoAs in which it makes the most difference.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源