But in most research fields, the importance of conference presentations pale in significance to final papers, published in peer-reviewed journals. (My understanding is that conference proceedings take on more weight engineering and computing.)
How much material from conferences is lost is relevant to discussions about the speed and efficiency of scientific communication, replication crises, file drawer problems, p hacking, and the permanence of the scientific record. It raises issues of how much you can trust what you see at conferences, and how soon you might be able to cite work that you have seen at a conference.
There is an emerging literature on this. There are nine journal articles on the topic in the last two years alone.
Because this is the poster blog, I’m most interested in how many posters eventually turn into papers. So far, I’ve found eight papers that estimated how many posters and oral talks were eventually published. Because I’m a
Posters have a significantly lower probability of being published than talks (t7 = -7.0, p = 0.00021). Posters have about a one in five chance of being published (21.2%), while talks have about a one in three chance of being published (34.4%).
Figure 1. Square = mean, horizontal line = median; box = 50% of data; whiskers = minumum and maximum; dots = individual papers (Bakkum & Trachimowicz 2015; Daruwalla et al., 2015; Durinka et al. 2015; Janssen et al., in press; Kinsella et al. 2015; Richling et al. 2014; Singh et al. 2015; Walsh et al. 2013).
The graph doesn’t quite give a real sense of the strength of the difference between posters and talks. Every one of the eight studies found that oral presentations were more likely to be published than posters at the same meeting. Here’s one that might show that a bit better:
Figure 2. Each line represents a single paper (same papers as Figure 1).
There are several hypotheses for why this difference exists, and they are not mutually exclusive.
- Posters may be more likely to be given by students, who might not stay in research careers. If they leave, nobody picks up writing up the project.
- Posters are more likely to be given early in a project, which means they are more likely to “blow up on the launchpad” or never be completed. Daruwalla and colleagues (2015) found no difference in the publication delay of posters and talks, however, which speaks against this hypothesis.
- The authors of the posters may self-select their work, presenting what they consider to be weaker or less interesting projects as posters and saving their “A game” for talks. Evidence from Sawatsky and colleagues (2015) supports this: they found that posters had a lower average rating of scientific quality than talks. (This might also explain why most people prefer talks over posters: experience has shown them talks are better.)
Another five papers calculated how likely an abstract was to become a publication, but did not separate posters and oral presentations. The average there was about the same as calculated for talks (37.7%), with ranging anywhere from 18.8% to 73.5%. The latter, for a veterinary conference, is an outlier; no other conference cracked 50%.
A few papers calculated or presented data on the time between the conference presentation and the final appearance of the publication. The delay is usually around the two year mark.
This conference presentation conversion rate might be a measure that researchers can use to convince tenure and promotion committees that they are likely to be productive in the future. If more than a third of your posters have been turned into papers in a certain time frame, you are definitely ahead of the game.
There doesn’t seem to be any hints of a relationship between the field of research or the size of the conference in whether a presentation becomes a paper. That said, a couple of papers did use the opportunity to claim that higher conversion rates were indicative of the “high quality” of particular conferences (Durawalla et al. 2015; Kinsella et al. 2015).
Of course, once I had a benchmark, I had to know if I was beating it.
I opened my CV and counted 36 conference posters in my academic career (excluding local meetings on my own campus). So far, 32 of those have been turned into publications, a very satisfying 88% conversion rate. But... this is a little misleading. Most of the studies had a time limit on getting those abstracts published. It was usually something like four or five years. If I knocked out papers that were five years or more between poster and publication (seven of them), my conversion rate drops to 69.4%. Which is... still not bad, actually. (But that one eight year delay between poster and paper... ugh.)
The ratio of posters to papers is nowhere near one to one, though. Sometimes projects made it onto multiple posters before being published as a single paper. One project got presented as a poster seven times before I was able to seal the deal and publish the project in a journal. My 32 poster presentations yielded 17 publications.
The reverse is true, too: there are some papers that I never presented at conferences, either as a talk or poster.
This topic seems to be an unexpectedly rich vein of meta-science. I would love to see an analysis from one of the mega-conferences, like the Neuroscience or American Geophysical Union meetings.
Extra special thanks!
This week’s post would have been impossible without the generosity of the awesome Biochem Belle! She first mentioned the poster to paper puzzle on Twitter. Then she saved me a bucket of work by creating a Google document with links to the papers below. Belle:
Additional, 9 April 2016: I ran a poll on Twitter asking people how many of their posters eventually turned into talks. I got 6o votes, which is not a bad sample size. Here are the results:
I am a little surprised by these results. These results are not exactly in line with the published results I summarize in the main part of the post. Maybe it reflects that scientists on Twitter are a more awesome subset of conference goers.
Update, 31 May 2016: I found a new paper which documents the lowest conversion from conference presentation to publication I have found yet. Disappointingly, it’s at a conference for medical publication professionals. Just 2.26% of posters turned into papers (3/133), and just 3.26% of talks became papers (1/31). Biochem Belle is also keeping track, and found another paper I missed. I have updated my box plot and t-test results accordingly:
Update, 19 March 2017: http://dx.doi.org/10.1503/cjs.010916, http://dx.doi.org/10.1111/vop.12464. (Placeholders until I read these.)
Update, 7 February 2019: Conservation biologists do a pretty good job of turning conference presentations into publications, but conversion of posters to papers (55%) still lags behind full-length oral presentations (61%).
The authors comment:
Our exploration of text titles revealed that podium presentations and posters may be tackling different study themes and topics within conservation. Thus, posters present research of good quality (likely to lead to publications) but on different topics. This calls for better appreciation of poster presentations within the range of scientific communications by both conference attendees and organizers.
Hat tip to David Shiffman. The big graph isn’t updated yet.
Update, 3 March 2020: I came across this systematic review that is probably the “last word” on the subject for the foreseeable future.
Scherer RW, Meerpohl JJ, Pfeifer N, Schmucker C, Schwarzer G, von Elm E. 2018. Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews (11): MR000005. https://doi.org/10.1002/14651858.MR000005.pub4
“Posters don’t turn into papers as often as talks” has been repeated so often that I am having a hard time seeing anything new or interesting come up from people scraping conference abstract books and looking them up a few years later. (Even though I know people will keep doing it.) A different approach is needed to dig into why posters don’t turn into papers. We need to survey authors of “orphaned” posters.
A reader forwarded these references, which both show the “posters don’t turn into publications as much as talks trend.”
Gandhi D, McLean I RW, Laiyemo AO. 2016. Analysis of the publication rate of the abstracts presented at a national gastroenterology meeting after 6 years. Digestion 94(4): 215-221. https://doi.org/10.1159/000450785
Scholcoff C, Sanghani P, Jackson W, Egloff HM, Sawatsky AP, Jackson JL. 2018. Peer review of abstracts submitted to an internal medicine national meeting: Is it a predictor of future publication? Journal of General Internal Medicine 33(7): 1002-1003. https://doi.org/10.1007/s11606-018-4416-8
Update, 1 May 2023: Another paper reports that talks are more likely to be published than posters. Again.
Issa TZ, Lee Y, Lambrechts MJ, Reynolds C, Cha R, Kim J, Canseco JA,
Vaccaro AR, Kepler CK, Schroeder GD, Hilibrand AS. Publication rates of
abstracts presented across six major spine specialty conferences. North American Spine Society Journal (NASSJ): in press. https://doi.org/10.1016/j.xnsj.2023.100227
This pattern has been demonstrated so many times I am reaching the point where I am tempted to say, “No further research needs to be done.”
Related posts
Variations on a theme: crayfish nociception
References
Bakkum BW, Trachimowicz R. 2015. Publication rates of abstracts presented at the 2006 meeting of the American Academy of Optometry. Optometry and Vision Science 92(11): 1069-1075. http://dx.doi.org/10.1097/OPX.0000000000000712
Carey LC, Stretton S, Kenreigh CA, Wagner LT, Woolley KL. 2016. High nonpublication rate from publication professionals hinders evidence-based publication practices. PeerJ 4: e2011. https://doi.org/10.7717/peerj.2011
Chand V, Rosenfeldt FL, Pepe S. 2008. The publication rate and impact of abstracts presented at the Cardiac Society of Australia and New Zealand (1999–2005). Heart, Lung and Circulation 17(5): 375-379. http://dx.doi.org/10.1016/j.hlc.2008.02.005
Daruwalla ZJ, Huq SS, Wong KL, Nee PY, Murphy DP. 2015. “Publish or perish”—presentations at annual national orthopaedic meetings and their correlation with subsequent publication. Journal of Orthopaedic Surgery and Research 10(1): 1-8. http://dx.doi.org/10.1186/s13018-015-0203-y
Durinka JB, Chang P-N, Ortiz J. 2014. Fate of abstracts presented at the 2009 American Transplant Congress. Journal of Surgical Education 71(5): 674-679. http://dx.doi.org/10.1016/j.jsurg.2014.02.002
Dyson DH, Sparling SC. 2016. Delay in final publication following abstract presentation: American College of Veterinary Anesthesiologists Annual Meeting. Journal of Veterinary Medical Education 43(1): in press. http://dx.doi.org/10.3138/jvme.33.1.145
Fosbøl EL, Fosbøl PL, Harrington RA, Eapen ZJ, Peterson ED. 2012. Conversion of cardiovascular conference abstracts to publications. Circulation 126(24): 2819-2825. http://circ.ahajournals.org/content/126/24/2819.abstract
Janssen T, Bartels R, Lind B, Villas Tome C, Vleggeert-Lankamp CLA. Publication rate of paper and podium presentations from the European Section of the Cervical Spine Research Society Annual Meeting. European Spine Journal: in press. http://dx.doi.org/10.1007/s00586-016-4404-9
Kalkan A, Kose O, Bilir O, Ersunan G, Ozel D, Guler F. 2015. Publication rate of abstracts presented at the emergency medicine congresses held by the European Society for Emergency Medicine (EUSEM) in 2011 and 2012. Emergency Medicine Journal 32: 728-732 http://dx.doi.org/10.1136/emermed-2014-204063
Kinsella SD, Menge TJ, Anderson AF, Spindler KP. 2015. Publication rates of podium versus poster presentations at the American Orthopaedic Society for Sports Medicine Meetings: 2006-2010. The American Journal of Sports Medicine 43(5): 1255-1259. http://dx.doi.org/10.1177/0363546515573939
Richling SM, Rapp JT, Funk JA, D’Agostini J, Garrido N, Moreno V. 2014. Low publication rate of 2005 conference presentations: Implications for practitioners serving individuals with autism and intellectual disabilities. Research in Developmental Disabilities 35(11): 2744-2750. http://dx.doi.org/10.1016/j.ridd.2014.07.023
Rosmarakis ES, Soteriades ES, Vergidis PI, Kasiakou SK, Falagas ME. 2005. From conference abstract to full paper: differences between data presented in conferences and journals. The FASEB Journal 19(7): 673-680. http://dx.doi.org/10.1096/fj.04-3140lfe
Sawatsky AP, Beckman TJ, Edakkanambeth Varayil J, Mandrekar JN, Reed DA, Wang AT. 2015. Association between study quality and publication rates of medical education abstracts presented at the Society of General Internal Medicine Annual Meeting. Journal of General Internal Medicine 30(8): 1172-1177. http://dx.doi.org/10.1007/s11606-015-3269-7
Singh A, Solanki P, Mishra D. 2014. Publication rate of scientific papers presented at the XXVI Annual Convention of National Neonatology Forum (NEOCON 2006). The Indian Journal of Pediatrics 82(1): 25-28. http://dx.doi.org/10.1007/s12098-014-1475-7
Verde Arregoitia LD, González‐Suárez M. From conference abstract to publication in the conservation science literature. Conservation Biology: In press. https://doi.org/10.1111/cobi.13296
Walsh CM, Fung M, Ginsburg S. 2013. Publication of results of abstracts presented at medical education conferences. JAMA 310(21): 2307-2309. http://dx.doi.org/10.1001/jama.2013.281671
I'm really surpised by the low conversion rates, because like you, my own conversion rate for posters would be close to (but not quite) 100% - and it wouldn't be much different for talks vs. posters. The data speak, though, so I must be an outlier. I suspect by far the biggest reason for the talk/poster difference is your #2: posters are more likely early in a career. I'll suggest another: posters may work better for presenting new methods (as opposed to results) and those might never be captured in the data as leading to publications, even if the method presented is used in dozens of later papers. (I cover this and a lot of other poster-vs.-paper considerations here: https://scientistseessquirrel.wordpress.com/2016/03/15/for-your-next-conference-poster-or-talk/)
ReplyDeleteThanks for digging into the data, Zen!
ReplyDeleteThe community has been discussing concerns & questions about publishing, and I think posters emphasize some of them (eg, publication bias, "weaker" studies).
Relevant to your & Stephen's conversion rates, it's probably worth noting that the references here mostly cover medical specialties. Might things be different in other disciplines?
It would be interesting to see the data on larger conferences, but I can also see the difficulty of parsing data. In biomed, we basically never cite posters, certainly not those that become papers, and we may change titles dramatically.
Finally there's another question that pops up: How well do findings in posters represent findings that are ultimately published? It's a complex issue, of course, because there are plenty of reasons results & conclusions might change. But here's an example on discrepancies between posters & final publications for systematic reviews:
http://dx.doi.org/10.1016/j.jclinepi.2015.03.006
Thank you so much for the amusing post i really appreciated it. If anyone wants the SEo services in India then visit then our website. http://goo.gl/VVYrCD
ReplyDeleteInteresting post!
ReplyDeleteI suspect your explanation that limited follow-up time has something to do with it makes a lot of sense. It's certainly true that posters are often presented while a project is still ongoing, and it may not be unreasonable to expect a delay of years rather than months before the paper is published.
I think it would also be interesting to see how this has changed over time. We know that the completeness of publication of clinical trials has improved massively in recent years. Back in the 90s, it was very common for negative trials not to be published. Putting a figure on that is not easy, but the All Trials favourite statistic that only half of trials are ever published was probably a reasonable estimate for that period.
However, more recent studies of completeness of publication show disclosure rates more in the range of 80-90%.
It may well be that research being at conferences today is much more likely to be published, but of course we won't know that for a while.
Still, it would be interesting to see how the most recent studies that we do have compare with some of the older ones.