Poster sessions may reflect scientific progress more accurately than the peer-reviewed literature

 I was listening to a podcast recently, and heard this:

In an analysis of more than 300,000 scientific conference presentations, informal posters or talks that scientists often endeavour to turn into papers, fewer than 40% were published in peer-reviewed journals, and negative or null findings were far less likely to be published than positive results.

I dug up the article this was referring to, and it was one I read before (Scherer et al. 2018). It’s cited the Better Posters book^*. The fact doesn’t surprise me, because the “file drawer problem” of selective publication has been known for decades. But hearing it in this context prompted a new thought.

Conference posters may represent the most accurate view of the progress of science available.

I contend this is kind of important. There is a huge amount of effort that goes into cataloguing and analysing peer-reviewed journal articles. There is an entire research field now of bibliometrics and industries built on providing data about scientific journal articles. 

And people have this expectation that the scientific literature should be “pure.” There are a lot of tears shed and many grumbling posts about the “pollution” of the peer-reviewed literature by incorrect studies, low-quality studies, and now generative AI. Listen for it, and you’ll hear the metaphors about cleanliness and purity come up all the time with regards to journal articles.

Why is there so much concern about that we know that the publication process filters results in a biased way? Biased towards statistically significant results, unexpected results, and so on.

I am not saying that all results are equally interesting to working professionals or that every data point is sacred. There is already too much scientific information in many fields for people to stay on top of everything, and filtering is not only necessary, but valuable.

But what I’m talking about here are specifically people who are interested in the big pictures, the trends. For instance, you might want to know not only what people say is working, but what people can’t get to work, no matter how many labs are trying. 

This is another reason why archiving conference posters matters, and why we should treat posters are part of the scientific record rather than ephemera.

* Still available and I’d love it if you got a copy or asked your library to get a copy!

References

Scherer RW, Meerpohl JJ, Pfeifer N, Schmucker C, Schwarzer G, von Elm E. 2018. Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews (11): MR000005. https://doi.org/10.1002/14651858.MR000005.pub4

External links

Audio long read: So you got a null result. Will anyone publish it? (podcast version)

So you got a null result. Will anyone publish it? (print version)

Picture from Origin of the Texas Sharpshooter

Link roundup for August 2024

It’s been a minute on the blog! I haven’t finished with this project, just been busy!

Shira Joudan writes about the transition from attending conferences as a trainee to attending as a supervisor:

I tried to remember what I didn’t know before attending my first conference and tried to be clear about how things work. Even though I am not that old, I sometimes forget what it is like to do some of these things for the first time, so I am sure I didn’t cover it all. I edited abstract drafts, explained what sessions at conferences are (and which were appropriate fits!), and then, eventually, we held practice presentations. I instructed poster presenters to expect interruptions and to try to prepare a quick explainer of their poster — and reassured them that preliminary results are completely acceptable at this meeting.

 What are your best suggestions for a new supervisor taking students to a conference for the first time?

• • • • •

Wolfe and Reineke provide user testing research on poster design. And it provides evidence to something I have been saying for years: Abstracts on posters are not necessary!

(T)he traditional, abstract format was rated as less usable than the other two formats and was the least preferred format. The sentence heading (without abstract) and #BetterPoster format were  rated equivalently in terms of usability and preference.

You may need a library subscription to read this one.

• • • • •

Here is a gallery of posters from the International Statistical Literacy Project. Lots of interesting posters to browse. I am, for instance, curious if this poster from Mongolia is truly meant to be printed and mounted on a board: 

I haven’t seen many posters that tall and skinny. Hat tip to Yaning Wu!

• • • • •

Miguel Balbin talks about choosing the font size for your poster:

Body Text: 30-36 pt – Legible from about a metre away.

This is better than many, suggesting a size that is about 50% bigger than many others recommend. But I always recommend that if you can go bigger, go bigger.

• • • • •

Ching-Ye Tien finds students in Taiwan are very positive about digital poster formats. A “digital poster” is described as being on a large screen or projected, not just something presented in a Zoom call. The students list these pros of digital posters:

• Seen as environmentally friendly due to less paper use.
• Easy access to information. (Not sure how this is judged).
• Cheap and fast.
• Easy to create and modify the content.