Baffling bibliometics

The prestige of publication has traditionally been based on citation-based metrics, most commonly the journal impact factor [1]. However, over the last decade, a new field of study has expanded rapidly across the scientific landscape—bibliometrics—a set of quantitative tools used to measure and analyse various aspects of academic publications [2]. For clinical research, bibliometrics helps evaluate research impact, dissemination, collaboration, and emerging trends [3].

The quantitative assessment of the scientific literature is changing rapidly with the creation of new evaluation tools, parameters, and normative data. The Internet has radically changed the speed, flow, and sharing of medical information. In addition, the exploitation of social media, along with the development of popular professional and scientific websites and blogs, has resulted in the emergence of new and alternative metrics, known as altmetrics [4]. What value do these currently have in scientific research beyond researchers promoting themselves?

Below are five key features of bibliometrics for clinical publications, summarised in our recent Insider's Insight [5].

Citation Analysis

• Definition:Measures the number of times a clinical publication is cited by other researchers. Citations are widely regarded as indicators of the impact or influence of a research work within the academic community [6].
• Importance in Clinical Research:High citation counts can indicate the relevance and importance of clinical studies to ongoing research or practice, signalling that the work contributes significantly to advancements in clinical knowledge [6].

Impact Factor (Journal-Level Metric)

• Definition:The average number of citations received per paper published in a journal during a specified period. It is often used as a proxy measure of the influence or visibility of a journal [2].
• Importance in Clinical Research:Clinical researchers often strive to publish in high-impact-factor journals because this can enhance the visibility and perceived credibility of their research. However, the impact factor was originally designed to assist librarians with journal selection rather than to evaluate individual articles or researchers [2][7].

H-Index (Author-Level Metric)

• Definition:A metric that attempts to measure both the productivity and citation impact of an individual researcher’s publications. It is calculated by identifying the maximum number h where the researcher has published h papers, each of which has been cited at least h times [8].
• Importance in Clinical Research:The h-index is widely used to assess the career impact of a clinical researcher, and high h-index values are often viewed as indicators of influence within a field [8].

Altmetrics (Alternative Metrics)

• Definition:Non-traditional metrics that measure the online impact of clinical publications, including social media mentions, news coverage, blog discussions, downloads, bookmarking activity, and policy document citations [5][9].
• Importance in Clinical Research:Altmetrics provide a broader view of the immediate impact of clinical research on public discussion, policymaking, education, and clinical practice beyond academic citations alone. They highlight how research is shared and discussed in public spheres and may capture attention long before citation data accumulate [9][10].

Co-Authorship and Collaboration Networks

• Definition:Analysis of the relationships between authors and institutions based on co-authorship patterns and collaborative publication activity [11].
• Importance in Clinical Research:Co-authorship analysis can identify influential researchers, research groups, and institutional partnerships. Collaboration networks may facilitate knowledge exchange, increase research productivity, and are often associated with higher-impact publications [11].

A recent article in Clinical Spine Surgery explored the evolution of current research-impact metrics and examined the evolving role of altmetrics in measuring the wider impact of research [12]. The authors suggest that altmetrics should form part of an informed assessment process alongside traditional bibliometric indicators. Does this create an opportunity for someone’s mother to tell the world how wonderful their research is, or is it a valid tool for assessing the influence and dissemination of scientific publications? Should we be concerned in a post-truth era?

References

1. Garfield E. The history and meaning of the journal impact factor. JAMA. 2006;295(1):90–93.
2. Pritchard A. Statistical bibliography or bibliometrics? Journal of Documentation. 1969;25(4):348–349.
3. Hood WW, Wilson CS. The literature of bibliometrics, scientometrics, and informetrics. Scientometrics. 2001;52(2):291–314.
4. Priem J, Taraborelli D, Groth P, Neylon C. Altmetrics: A Manifesto. 2010.
5. Niche Science & Technology Ltd. Bibliometrics Breakdown: An Insider's Insight. 2016.
6. Bornmann L, Daniel HD. What do citation counts measure? A review of studies on citing behaviour. J Doc. 2008;64(1):45–80.
7. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314(7079):498–502.
8. Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569–16572.
9. Priem J, Piwowar HA, Hemminger BM. Altmetrics in the wild: Using social media to explore scholarly impact. arXiv. 2012.
10. Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J Med Internet Res. 2011;13(4):e123.
11. Katz JS, Martin BR. What is research collaboration? Res Policy. 1997;26(1):1–18. doi:10.1016/S0048-7333(96)00917-1.
12. Butler JS, Sebastian AS, Dicker P, et al. The Evolution of Current Research Impact Metrics: From Bibliometrics to Altmetrics? Clin Spine Surg. 2017;30(5):226–228.