The number game: 7 Global trends in impact factor and other citation metrics
Global science
Science is global and growing at an immense pace. This series provides an overview of the status of science and research around the world, in terms of R&D expenditure, citation metrics, collaboration, and internationalization.
This is an exciting time for scholarly publishing with an exponential growth in scientific progress and knowledge expansion in the recent years. As scientific literature continues to expand and become more accessible, funders and researchers are attaching more importance to evaluating the academic impact of this growing body of publications. Therefore, research metrics and analytics based on metadata, such as, citations, alternative metrics, patterns of data usage, etc. are the focus of all major stakeholders. Although citations and impact factor (IF) continue to be strongholds of validation of research output, research metrics are constantly innovated upon because funders, researchers, libraries, and institutes seek new metrics that can either replace or supplement established metrics.
Understanding the citation patterns and the trends in the use of impact metrics is vital to gaining insights into the dynamic landscape of scholarly publishing. According to a recent report by The International Association of Scientific, Technical and Medical Publishers (STM), an international trade association organized and run for the benefit of scholarly, scientific, technical, medical and professional publishers, here are 7 noteworthy trends in citations and impact factor.
1. Growing proportion of citations from outside the country of authorship
Citations are central to academic writing as they direct the reader to additional and more elaborate discussion pertaining to a paper’s topic. The 2015 STM report indicates that the number of citations has increased in comparison to the number of articles published between the year 2003 to 2008: the number of publications increased by 33%, while citations increased by 55%. Globally, the average number of citations per article has increased from 1.7 in 1992 to 2.5 in 2012. Some of the reasons for this trend are:
- Growth of the literature
- Growth in coauthorship
- Longer reference lists
Citation inflation: Increase in the average citations per article, by country of author
Source: 2015 STM report
Notably, globalization of research has an impact on citation patterns; there is an increase in the proportion of citations from outside the country of authorship. Due to international coauthorship and a growing awareness of international research, international citations (citations to an international journal from countries other than the author’s home country) have gradually increased for all major scientific countries in the last two decades. An exception to this is China where the proportion of international citations dropped from 69% in 1992 to 49% in 2012, which suggests that China’s expanding article output is being used mostly within China.
2. Articles published in a high IF journal can be of different quality
A journal’s IF is a measure of the frequency with which the "average article" in a journal has been cited in a particular period. Hence, a high IF journal is widely believed to comprise of high quality articles. However, the STM report states that only 15% of articles accounted for 50% of citations, and 90% of citations were generated by 50% of articles. Thus, the top articles in a journal may receive 9 times as many citations as the bottom half, and the journal can have many uncited articles too.
3. Citation practices vary between disciplines
The citation patterns vary largely between research disciplines primarily because of the differences in the incidence of coauthorship. Fundamental and pure subject areas tend to have higher average impact factors than specialized or applied ones. Hence, mathematics has an average IF of 0.556, while molecular and cell biology has an IF of 4.763. This variation can be so skewed that the top journal in one field may have an impact factor lower than the bottom journal in another area. Authors tend to refer to their own work and since the coauthorship average varies between research disciplines, the average levels of citations among different fields also vary considerably. However, this variation can be accounted for and corrected using field-weighted citations when comparing research across different fields.
4. Rising popularity of alternative metrics tracking article or author level impact
Despite the numerous shortcomings, journal IF is widely used as the yardstick of evaluating the impact of authors’ work. However, in recent times, additional metrics have been proposed to complement the metrics provided by citations and downloads to measure the overall impact of a piece of work better. Some of the most popular alternative metrics in use are Altmetric, the Source Normalised Impact per Paper (SNIP), SCImago Journal Rank (SJR), the h-index, and so on. There is also a growing interest in tapping the impact at the article or author level; Kudos and ImpactStory are two such initiatives. A challenge while using article-level metrics is that they can be easily gamed and hence cannot be relied on completely. Another method of calculating article level metrics is social media activity. Although the role of social media in dissemination of science is garnering attention, there is skepticism regarding its dependability in measuring genuine long-term impact.
5. Older papers are cited more frequently than in the past
As observed by a group of researchers from Google Scholar, there is a shift in citation behavior to the effect that authors are citing a higher proportion of older papers than in the past, and that highly-cited papers are more likely to be found in non-elite journals. The authors of the study believe that accessibility and discoverability of scientific literature due to the internet has played a role in the unearthing of older papers and papers published in non-elite journals.
6. Citations by patents measure wider impact
Patent citation, which refers to citations to scientific literature within patents, is considered as a measure of wider impact beyond academia because it indicates a strong correlation between a research paper and its role in innovation. Patent citations are typically much older than literature citations, mainly because of the delay in granting patents; for example, the NSF analysis looks at an 11-year window after a 5-year lag. In the US, the proportion of patents citing academic literature increased from 12% to 15% between 2003 and 2012. The majority of cited articles fall into three fields: biological sciences (48%), medical sciences (23%), and chemistry (11%).
7. Journal Usage Factor, an emerging metric system
Some experts believe that the number of downloads of scientific articles gives a better measure of an article’s wider impact than citations do. This is true particularly in the case of journals in medicine and other journals with a large practitioner readership. Hence, the project COUNTER (Counting Online Usage of Networked Electronic Resources) was initiated to calculate the Journal Usage Factor (JUF). JUF is the “median value of a set of ordered full-text article usage data” calculated by dividing the total number of article downloads by the number of articles published in a journal over a specified window of time.
Article downloads by country, 2010
Country | Article downloads (millions) | Proportion of global total (%) |
Global total | 1065 | 100.0 |
USA | 327 | 30.7 |
China | 105 | 9.9 |
UK | 100 | 9.4 |
Germany | 70 | 6.6 |
Japan | 62 | 5.8 |
Source: 2014 STM report
Number of downloads is a potentially useful complementary metric to citation data but cannot replace the traditional metrics as the two signify different things i.e., reader and author choices, respectively. Moreover, citations and downloads have different profiles over time: most downloads occur in a peak a few months immediately following publication, while citations build over a longer period of 2–3 years. While JUF’s use is mostly limited to library acquisition settings and for authors selecting journals to submit to, it is a useful metric to gauge the usage patterns of scientific papers.
As academe is expanding its reach and widening its horizons, it is shifting beyond the traditional metric systems to explore newer and better ways of measuring the impact of research. While these newer metrics serve to measure certain aspects of research impact, it is unlikely that they can be scaled up to replace the traditional metrics.
Published on: Jun 19, 2015
Comments
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.
