The Lost Science: Conferences and the Meeting the Missing Data

When asked to present on the future of the clinical trial, I spent 70 hours on research and slide preparation. After testing the presentation on our Niche team, I delivered it to an audience of 50–60 people. It is a sobering reality that with such a small audience, the meticulously crafted messages I poured my soul into are unlikely to be widely disseminated. This is, of course, nothing new. As scientists, we habitually pour considerable time and resources into posters and presentations that are swiftly forgotten, offering a dismal return on investment (ROI) [1].

Over the last 30 years, I have given over 100 presentations. The contents of just over 60% have made it into the scientific literature. Regarding the remaining 40%, there is little record beyond the dusty 250-word abstracts languishing in conference proceedings on my bookshelf. If you want to know more about that original research, you would need to track down one of the original authors, assuming the conference organizer hasn't archived the materials (which is unlikely unless the work was presented recently). This feels like a poor legacy for all the effort invested. It begs the question: is there something inherently wrong with conferences as a mode of scientific communication?

Most scientists regularly attend and present at conferences. Meeting to discuss concepts goes back eons, with the modern scientific meeting evolving from the London assemblies that led to the formation of the Royal Society in 1660. Initially, distance and opportunity kept meetings small, but larger gatherings proliferated with new scientific societies in the mid-1800s. Today, there are hundreds of scientific conferences globally every year.

Although the conference model might be criticized for its apparent inefficiency as a means of dissemination [1], it remains integral to scientific practice. Conferences are an arena where young scientists cut their teeth, serving as an informal peer review system that helps researchers refine their work [2]. Feedback obtained during coffee breaks often benefits the final published research. Furthermore, meetings allow researchers to hear about adjacent disciplines and make connections, creating opportunities for the serendipitous cross-pollination of ideas. Such is the messy, wonderful nature of scientific discourse.

Serendipity, Spreadsheets, and the "Jolly"

I have many positive experiences from attending conferences. For instance, my post-graduate supervisor was (very) slow to register me for my PhD. For 18 months, I worked without recognition. In frustration, I reached out to a group I had met at an MRC conference on a possible project we could work on together. That brief liaison led quickly to a publication [3], lifelong friendships, and collaborations spanning 25+ years. It also saw my supervisor register me for my PhD within days of me reporting the collaboration. This episode convinced me that every young scientist should use conferences to reach beyond their laboratory environments to discover new mentors and collaborators.

This is not just my experience. A 2004 survey of over 1000 conference attendees found that:

• 66%learned something that changed the direction of their research.
• Over halfindicated that something learned saved them time and money.
• 60%reported that attendance led to a new collaboration [4].

These observations echo William Barton Rogers (a founder of the AAAS) in 1848: "For us, such reunions of the scientific brethren... are of precious value... What new impulses to exertion, what encouragement and guidance do they not give?"

And there is the rub. Meetings not only include formal opportunities to interact with friends and colleagues but also involve informal interactions. Scientific meetings often include events such as dinners or receptions. Hollywood has done its best to promote the cliché that conferences are a "jolly"—in The Day After Tomorrow, a scientist quips that they are "all dancing girls, wine and parties."

Sadly, economists run the world, and while they benefit from serendipity, they struggle to factor its randomness into their spreadsheets. It has been suggested that the outcomes of conferences are incommensurate with our investment of time and money [1]. Research supports the view that the multi-billion dollar global expenditure on conferences often results in "lost research" that never reaches the public record [5]. Furthermore, the pressure to publish "positive" results in journals creates a bias, making the conference floor one of the few places where we share our failures, preventing colleagues from wasting time on fruitless concepts [6]. However, survey the attendees themselves, and the consensus is that face-to-face meetings, in an open environment, lead to progress [3]. It is difficult to conceive how sterile webinars could promote the same educational and growth benefits.

A Prescription for the Future: Databases, Proceedings, and a Dash of Courage

If you feel you aren't getting the most from conferences, review your approach [7]. You may be a pathological introvert after years in the lab. Make sure you utilize practical hacks to maximize your attendance [8][9]. If you are presenting, ensure your poster or slide deck best communicates the value of your science. If you are presenting then make sure your poster [10][11] or presentation [12] best communicates the value of your science.

As for the ‘lost science,’ keep engaging. Your interactions ensure your data isn't actually lost. But we need structural fixes. At the Association of Human Pharmacology in the Pharmaceutical Industry (AHPPI) and the European Federation for Exploratory Medicines Development (EUFEMED), we have adopted a policy of maximizing dissemination by publishing proceedings in Frontiers in Pharmacology [13][14]. And although conference proceedings are often seen as lesser publications, evidence points to our initiative being well received. Our earlier publication on the new first in human guidelines issued by the European Medicines Agency already having an altmetric score of 8, nearly 450 downloads and over 3,500 views [13].

Conferences are clearly an unpredictable method of disseminating knowledge in terms of getting the right information to the right people. Research suggests that we are not particularly talented at pre-selecting work and the presenters that we want to connect with [15]. This most likely becomes more relevant at larger-scale events, where the volume of information ‘on show’ and the number of people to meet in the time you have available to you starts to become challenging. It seems likely that we miss out on some of the ‘opportunities’ on offer. It is this apparent ‘unpredictability’ of conferences that has led some to question their value [1][15]. And once we raise the spectre of ‘value’ we are down to justifying the time taken to developing conference materials, fees for attendance as well as any travel and accommodation.

Sadly, economists run the world and, although they and delighted to benefit from serendipity, many find it difficult to factor its randomness into their spreadsheets. It has been suggested that the outcomes from conferences are incommensurate with our investment of time, effort and money and that the current conference model is unsustainable in a research world with only finite resources [1][15]. Fortunately, the very nature of science dictates that for every hypothesis there is a counter hypothesis. Survey conference goers themselves and you will find that the overwhelming consensus is that face-to-face meetings between conference attendees, in an open and sharing environment, leads to progress on many fronts [4]. It is difficult to conceive how more modern (sterile) webinars and teleconferences could promote similarly the educational and growth benefits derived through social interactions.

So, is there something wrong with conferences? Yes—they are inefficient, expensive, and wasteful. But is there something inherently right with them? Absolutely. Everyone benefits when we share our trials, tribulations and failures, something that can be difficult to do within the literature itself, biased as it is to publishing positive findings [16]. In short, conferences are the unpredictable, human engine of collaboration and progess. Just make sure you talk to a stranger.

References

1. Rowe NE. Why It Is Time to Make Conferences Worth It. The Scientist. 2019.
2. Smaldino PE, McElreath R. Open science and modified funding lotteries can impede the natural selection of bad science. R Soc Open Sci. 2019;6(7):190194.
3. Rutherford PA, et al. Sodium-lithium countertransport activity is not affected by short-term insulin exposure in vivo or in a physiologic medium in vitro. Metabolism. 1993 Sep;42(9):1087-9.
4. Aitken JW. What's the Value of Conferences? The Scientist. 2006.
5. Rowe NE. The Economic Cost of Attending Educational Conferences. Int J Soc Educ Sci. 2019;1(1):1-10.
6. Mlinarić A, Horvat M, Smolčić V. Dealing with the positive publication bias: Why you should really publish your negative results. Biochem Med (Zagreb). 2017;27(3):030201.
7. Hardman TC. (2019) Ghost, Zombie, Vampire or Werewolf? Which Conference Monster Are You and How Do You Escape It?
8. Hardman TC. (2018) The Introvert's Guide to Conferencing for Nerds – Top 10 Tips.
9. Hardman TC. (2018). 3-step hack to conference superhero. LinkedIn. 2019.
10. Niche Science & Technology Ltd. (2018). An Insider’s Insight into Poster Presentations
11. Hardman TC (2018). Top 5 tips when making posters.
12. Niche Science & Technology Ltd. (2019). Ta Dah! An Insider’s Insight into Creating Better Slide Presentations.
13. Reijntjes S, et al. The Association for Human Pharmacology in the Pharmaceutical Industry London Meeting 2018: Brexit and Other Challenges in Early Phase Drug Development. Front Pharmacol. 2018;9:1301.
14. Breithaupt-Grögler K, et al. The New First-in-Human EMA Guideline: Disruptive or Constructive? Outcomes From the First EUFEMED Discussion Forum. Front Pharmacol. 2019;10:398.
15. Rowe NE. 'Poster, poster, on the wall; were you even there at all?' A mixed method research into the efficacy and perceptions of conference poster presentations. Acta Universitatis Lapponiensis. 2019;254.
16. Mlinarić A, Horvat M, Smolčić V. Dealing with the positive publication bias: Why you should really publish your negative results. Biochem Med (Zagreb) 2017;27:030201.