Poster sessions may reflect scientific progress more accurately than the peer-reviewed literature

 I was listening to a podcast recently, and heard this:

In an analysis of more than 300,000 scientific conference presentations, informal posters or talks that scientists often endeavour to turn into papers, fewer than 40% were published in peer-reviewed journals, and negative or null findings were far less likely to be published than positive results.

I dug up the article this was referring to, and it was one I read before (Scherer et al. 2018). It’s cited the Better Posters book^*. The fact doesn’t surprise me, because the “file drawer problem” of selective publication has been known for decades. But hearing it in this context prompted a new thought.

Conference posters may represent the most accurate view of the progress of science available.

I contend this is kind of important. There is a huge amount of effort that goes into cataloguing and analysing peer-reviewed journal articles. There is an entire research field now of bibliometrics and industries built on providing data about scientific journal articles. 

And people have this expectation that the scientific literature should be “pure.” There are a lot of tears shed and many grumbling posts about the “pollution” of the peer-reviewed literature by incorrect studies, low-quality studies, and now generative AI. Listen for it, and you’ll hear the metaphors about cleanliness and purity come up all the time with regards to journal articles.

Why is there so much concern about that we know that the publication process filters results in a biased way? Biased towards statistically significant results, unexpected results, and so on.

I am not saying that all results are equally interesting to working professionals or that every data point is sacred. There is already too much scientific information in many fields for people to stay on top of everything, and filtering is not only necessary, but valuable.

But what I’m talking about here are specifically people who are interested in the big pictures, the trends. For instance, you might want to know not only what people say is working, but what people can’t get to work, no matter how many labs are trying. 

This is another reason why archiving conference posters matters, and why we should treat posters are part of the scientific record rather than ephemera.

* Still available and I’d love it if you got a copy or asked your library to get a copy!

References

Scherer RW, Meerpohl JJ, Pfeifer N, Schmucker C, Schwarzer G, von Elm E. 2018. Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews (11): MR000005. https://doi.org/10.1002/14651858.MR000005.pub4

External links

Audio long read: So you got a null result. Will anyone publish it? (podcast version)

So you got a null result. Will anyone publish it? (print version)

Picture from Origin of the Texas Sharpshooter