Beyond Impact Factors: Rethinking Research Value in the Age of Open Data

Editor’s Note – October is the month when we observe the International Open Access Week. The theme this year is “Who Owns Our Knowledge?” – and to address this, we bring to you a perspective from Yateendra Joshi…

What is the meaning of “community impact” in research?

To me, it simply means what it says, namely impacts of something – a piece of research, a social intervention, new infrastructure – on the community at large, and not necessarily limited to the community that was the intended beneficiary of that something.

Can journal impact factors measure community impact?

Journal impact factors can never be even a rudimentary measure of community impact. In fact, it will be foolhardy to reduce the impacts to a single number because they are far ranging both temporally and spatially, unpredictable, and indirect. Take the Green Revolution for example: the intervention was simply new varieties of wheat and rice, but the impacts merited the term ‘revolution’: high yields, of course, but along with greater demand for chemical fertilizers and pesticides, disappearance of dozens of varieties (‘land races’, to use a more technical term), increasing depletion of groundwater and pollution . . .

I will strongly recommend one recent book and one not so recent. The recent book, AI Snake Oil : what artificial intelligence can do, what it can’t, and how to tell the difference, shows that our faith is predictive AI is grossly misplaced, and Everything is Obvious: once you know the answer is good reminder that even well-meaning  attempts to predict the outcomes often fail totally because they, by definition, cannot account for randomness, which is everywhere.

Is there any actual metric for community impact?

Yes, we now have a tool not only to measure community impact but also to predict it:

“Wang and his colleagues built a tool called Funding the Frontier, which integrates data on research publications, patents, policy papers and clinical trials, and presents the information in a visually intuitive way.” – https://www.nature.com/articles/d41586-025-03120-6

What changes are needed in academic publishing to value community impact alongside traditional metrics?

I will question the ‘alongside’ part because I believe that the traditional metrics have derailed the whole ecosystem of scholarly publishing: what was once meant to produce and disseminate knowledge has been reduced to pursuit of metrics simply because the metrics offer convenience and eliminate deep thinking, the need to be patient and have an open mind, and the joy of science. What we need is the willingness to measure the metrics themselves: to examine whether they really quantify the quality of scholarship—whether that quality is indeed quantifiable. And then to take a deep dive into retrospectively ascertaining whether proven breakthroughs in science could have been captured and highlighted by those metrics.

Lest this should sound negative, I should add that the increasing insistence on making the data available – after all, this is the age of open data – is a good thing and may even qualify as a community impact because the data can be put to many uses well beyond the purpose for which the data were collected in the first place. And not always for nefarious ends. Climate change is a good example: all sorts of data from a range of disciplines are converging to the inescapable conclusion that climate change is a reality.

What do you think about the way research impact is measured today? Do you think impact can truly be measured? What are some challenges and opportunities you think we will face in the next decade of research communication? Chime in with your questions and engage with the Editage Insights community in our Q&A forum!