European science organizations reach an agreement on long-due research assessment reformation
With a shift in the research landscape and ecosystem, an overhaul of the criteria used to assess research is long overdue. The current research assessment system1 tends to tip in the favor of certain performance metrics, such as publishing in high impact factor journals, citation counts, h-indexes, patent registrations, and securing grants. Having such narrow criteria2 for research assessment can put pressure on researchers and has amplified concerns around research ethics and integrity. Some initiatives like the SCOPE framework3 and the San Francisco Declaration on Research Assessment4 have been working towards improving, implementing, and promoting best practices in assessing researchers’ contributions and outputs.
The Agreement on Reforming Research Assessment, the final version of which was announced on 20 July 2022, is being viewed as a promising step towards meaningful change. Four years in the making, the initiative is a result of inputs from 350+ organizations that represent the diverse research communities across Europe to create inclusive evaluation systems. Having been facilitated by the European Commission, the initiative aims to go global with their vision.
What is the Agreement for Reforming Research Assessment?
The Agreement on Reforming Research Assessment5 offers a path towards changing the current narrow criteria used to assess6 research, researchers, and research organizations. Over 350 organizations from more than 40 countries7 contributed their ideas and feedback to the draft agreement put forward by a team of representatives from the European University Association, Science Europe, and the European Commission, along with Dr Karen Stroobants’s contribution as an individual with expertise in research on research. The participating organizations included a diverse set of stakeholders including universities, research institutes, scholarly societies, research funders, accreditation and evaluation agencies, and national and regional authorities, thus offering a diverse pool of perspectives.
The agreement details the vision of reform, including the principles, commitments, and timeframes. The agreement pertains to the assessment of research organizations and research units, research projects, individual researchers, as well as research teams8. While the proposed assessing bodies include groups like assessment authorities, organizations that fund and perform research, and prize awarding organizations.
The agreement aims to put forth principles and guidelines1 that all signatories have agreed on following during research assessment. Establishing a uniform research assessment model is not the goal; rather, the agreement aims to promote practices that ensure unbiased assessments by maintaining transparency when making any decisions. Of the ten commitments listed by the documents, the four core ones include:
- Recognizing the diversity of research contributions and careers, and rewarding researchers for mentoring, supervising, and teaching
- Using qualitative evaluation, as the main basis for research assessment, shifting focus from quantity to quality
- Doing away with research assessment that inappropriately uses publication-related and journal-based metrics, specifically related to h-index and journal impact factors
- Avoiding evaluations based on the prestige of research organizations
The other six supporting commitments are aimed at helping the signatories in implementing the new guidelines in their work.
What’s next?
The agreement will be open for signatures9 from 28 September 2022, which will be launched at the Research and Innovation Days, with organizations and institutions performing research assessment encouraged to sign on. The signatories coming together will form a coalition and meet later this year for their first general assembly meeting. The coalition will review the progress made and build a more permanent and sustainable structure, one of the aspects of which will be to give researchers, particularly early-career researchers, instrumental roles. This would allow early-career researchers to be a part of the assessment system, along with their institutions, research funders and other senior researchers.
The Agreement on Reforming Research Assessment, which stems from months of negotiations, can play a significant role in promoting high-quality and high-impact research and in shifting away from a “publish or perish” academic culture.
References
- Support Europe’s bold vision for responsible research assessment. Nature News. https://www.nature.com/articles/d41586-022-02037-8
- Curry, Stephen; de Rijcke, Sarah; Hatch, Anna, et al. (2020) The changing role of funders in responsible research assessment: progress, obstacles and the way ahead (RoRI Working Paper No.3). Research on Research Institute. Report. https://doi.org/10.6084/m9.figshare.13227914.v2
- SCOPE framework for research evaluation. https://inorms.net/scope-framework-for-research-evaluation/
- San Francisco Declaration on Research Assessment. https://sfdora.org/read/
- Agreement on Reforming Research Assessment. European University Association. https://www.eua.eu/downloads/news/2022_07_19_rra_agreement_final.pdf
- The Agreement on Reforming Research Assessment is now final. Science Europe. https://www.scienceeurope.org/news/rra-agreement-final/
- List of organisations having expressed interest in being part of a coalition on reforming research assessment. https://research-and-innovation.ec.europa.eu/system/files/2022-07/ec_rtd_call-for-interest-ra.pdf
- EU's Agreement on Reforming Research Assessment. Aalto University. https://www.aalto.fi/en/research-art/eus-agreement-on-reforming-research-assessment
- Reforming research assessment: The Agreement is now final. European Commission. https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/reforming-research-assessment-agreement-now-final-2022-07-20_en
Comments
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.
