I got an email from ResearchGate this week. A manuscript I wrote with colleagues back in 2021 had reached a milestone [1]: apparently it had been cited more than 50 times.
There it was, a simple bit of algorithmic feedback. A tiny pleasure. Scientists like to pretend that citation counts are of little consequence, that we toil away for the love of knowledge, not for a number. And yet, I suspect most researchers recognise the little psychological lift that comes from discovering that something they wrote a few years ago is still being read, discussed and cited. It suggests that the work has, in some small way, become part of the ongoing conversation.
For many of us, ResearchGate has become something like a ‘LinkedIn for scientists.’ We create profiles, list our publications, follow colleagues, share papers and, in turn, receive a continuous stream of notifications about reads, citations and other milestones. The site provides a constant reminder that our scientific manuscripts have a life after publication. It is worth remembering, however, that ResearchGate's citation statistics are based on its own indexing. A paper's citation count on ResearchGate is unlikely to be identical to numbers in Google Scholar, Scopus or Web of Science. ResearchGate scoring is based on a smaller, more selective database [2]. Studies have demonstrated that citation counts vary substantially between databases due to differences in their coverage and indexing methods [3][4]. One study even found that Google Scholar often identifies more early citations than ResearchGate [2]. So, 50 citations on ResearchGate sounds good, but it is very individual and limited metric.
This email prompts a more interesting, and challenges, question: what does the score actually mean when a paper has been cited 50 times?
At first glance, the answer seems obvious. Fifty other publications have cited the work. You might say our paper has entered the scientific conversation. But a citation is not necessarily an endorsement. A paper may be cited because another researcher agrees with its findings, but it may just as easily be cited because it provides useful background, describes a useful methodology, offers a convenient definition, or is a foundational reference for a particular field. It might also be cited because another paper is actively contrasting or disputing its findings. Occasionally, it is simply copied from a previous paper’s reference list without much consideration. Thus, a crucial distinction must be made between being cited and being truly influential. The simple act of being mentioned is not a qualitative measure of a paper's value or impact.
Why did citations become such an important proxy for scientific influence? Clearly, they provide a relatively simple quantitative measure of citation by other published research. They are, in theory, evidence that the research remains relevant to the current publishing landscape. However, the limitations are well-documented [5]. We have written our own Insider’s Insight about them [6].
Citation behaviour differs substantially between disciplines. Some fields generate many citations because their reference lists are extensive, whereas other disciplines have much lower citation rates. Citation rates also depend heavily on the age of the paper, the size of the research field, the popularity of the subject, the journal, its readership, and whether the paper is open access. Therefore, 50 citations shouldn’t be interpreted in isolation. A 50-citation paper in one discipline may represent a very different level of influence from a 50-citation paper in another. This is why bibliometricians often advocate for field- and time-normalised citation metrics, such as percentiles, which provide a more contextualised view of a paper's relative performance. Is it in the top 1% or the bottom 50% of its field? Raw numbers, without this context, are almost meaningless.
Returning to my ResearchGate email, I find myself exploring the curious psychology of being told that something I wrote years ago has reached a numerical milestone. Why do we care about these numbers?
My feeling is that these provide a form of delayed feedback. Scientific publishing is unusual. We spend months or years conducting research, analysing data, and writing a manuscript, only to release it into the world with little immediate evidence that anyone has actually read it. A citation, years later, is a small signal that the work has not disappeared into the academic void. It is substantially more consequential than a ‘like’ or a ‘share’ on social media, but it still does not tell us whether the underlying science was good. But this is where scientists tend to let themselves down. Publication should never be the end of the process. We also need to promote our work [7].
This brings us to the concept of Altmetrics, another attempt to understand what happens to research after publication. Citations are only one form of attention. The Altmetric Attention Score is an automatically calculated, weighted count of the attention detected for a research output from sources such as news outlets, blogs, social media, and policy documents [8]. The weightings reflect the relative importance of different sources; a news article contributes more than a social media mention.
Altmetric will tell you that your work appears in a certain percentile. This provides context, indicating how an output compares with other research outputs within its relevant comparison set [8]. However, the more important question is this: does being in the 95th percentile mean the paper is scientifically better? Clearly, the answer is no. The Altmetric Attention Score measures attention, not scientific quality [9]. A controversial or surprising paper may attract enormous attention without being scientifically important or correct. Conversely, an excellent but highly specialised paper may have very little online attention while being extremely valuable to the small scientific community that uses it. This is a central argument: attention is not the same as impact, and impact is not the same as quality.
So, what does reaching 50 citations actually say about the scientist who wrote the paper? It says something, but much less than we might assume.
It suggests the research has achieved some degree of visibility. It may suggest that the question addressed was relevant to other researchers. It may indicate that the paper provided useful information, methodology, or context. But it does not establish that the scientists themselves were exceptionally talented, that the paper was good, or that the scientists made an important contribution to the scientific narrative. Likewise, a paper with few citations does not necessarily indicate poor science. It might be a meticulous replication study or a robust negative finding, both vital to the scientific process but rarely celebrated.
This is the uncomfortable reality. Citation metrics have become deeply entangled with academic careers, promotion, funding, and perceptions of prestige [6][7]. The h-index is a prime example of how citation behaviour has been converted into a measure of researchers themselves. A metric originally intended to report on patterns of scholarly attention has gradually become treated as a proxy for the worth of scientists [5].
The solution lies in taking these data with a pinch of salt and adopt a more responsible use of metrics. The San Francisco Declaration on Research Assessment (DORA) [10] and the Leiden Manifesto [11] provide clear principles for this. The key argument is that quantitative indicators should support, rather than replace, qualitative expert assessment. Each individual piece of research should ultimately be assessed on its own merits, not on the journal in which it was published or the number of citations it accrued. Such perversion of the publishing landscape can on serve to bias the scientific narrative – where the ‘best’ research is often unavailable to researchers, locked behind journal paywalls [12].
Scientific contribution can take many forms that citation counts cannot adequately capture: the development of new methods, the generation of data, the execution of replications, clinical translation, influence on policy, and contributions to collaborative research. These are all vital contributions that may not be reflected in a citation count.
Returning to my 2021 paper [1], I will not pretend the 50-citation milestone was meaningless. It was genuinely pleasing. But perhaps the pleasure comes not from the number ‘50’ itself, but from what it represents: somewhere, sometime, another scientist found our work sufficiently relevant to include it in their own. It suggests that our efforts were not in vain. That is a much more meaningful interpretation of a citation. Scientific papers are strange things. We spend enormous amounts of time writing them, release them into the world. And then, those that don’t believe in managing their dissemination largely lose control of what happens next. Reach out to mw if you would like to learn how to boost your citation and altmetric scores. A notification years later can suddenly remind us that the words we wrote are still travelling through the scientific literature. Perhaps the most satisfying research metric is not how many people noticed the paper, but whether it became useful to someone.
References
Get our latest news and publications
Sign up to our news letterResources
Social
Contact us
Address
Niche Science & Technology
Unit 26 Falstaff House
Bardolph Road
Richmond TW9 2LH
United Kingdom
