Most people know that when they show an average, there should be an indication of how much smear there is in the data. It makes a huge difference to your interpretation of the information, particularly when glancing at the figure.
For instance, I’m willing to bet most people looking at this...
I’m likewise willing to bet most people looking at this (which plots the same averages)...
Would say, “There’s so much overlap in the data, there might not be any real difference between the control and the treatments.”
The problem is that error bars can represent at least three different measurements (Cumming et al. 2007).
- Standard deviation
- Standard error
- Confidence interval
Sadly, there is no convention for which of the three one should add to a graph. There is no graphical convention to distinguish these three values, either. Here’s a nice example of how different these three measures look (Figure 4 from Cumming et al. 2007), and how they change with sample size:
I often see graphs with no indication of which of those three things the error bars are showing!
And the moral of the story is: Identify your error bars! Put in the Y axis or in the caption for the graph.
ReferenceCumming G, Fidler F, Vaux D 2007. Error bars in experimental biology The Journal of Cell Biology 177(1): 7-11. DOI: 10.1083/jcb.200611141
A different problem with error bars is here.
Thanks for posting on this very important, but often ignored, topic! A fundamental point is also that these measures of dispersion also represent very different information about the data and the estimation. While the standard deviation is a measure of variability of the data itself (how dispersed it is around its expected value), standard errors and CI refer to the variability or precision of the distribution of the statistic or estimate. That's why, in the figure you show, the SE and CI change with sample size but the SD doesn't: the SD is giving you information about the spread of the data, and the SE & CI are giving you information about how precise is your estimate of the mean.
ReplyDeleteThus, not only they affect the interpretation of the figure because they might give false impressions, but also because they actually mean different things! This makes your take-home message even more important: Identfy your error bars, or else we can't know what you mean!
A rule of thumb I go by is: if you want to show how variable data are, you should show SDs; if you want to show how confident you are about something you're estimating, or the difference between estimates such as means, which is often the goal of a statistical test, you should show SE or (preferably) CIs.
OR, if you N is small enough, put the data right on the graph instead of plotting summary statistics.
ReplyDeleteI think the real lesson of this post is, always choose the standard error, it will make your error bars look smaller ;-)
ReplyDeleteNothing sensible to say except I know two of the three authors, and share a friend with the third lead author...
ReplyDeleteNote that in PNAS Information to Authors (http://www.pnas.org/site/misc/iforc.shtml), under "Figure Legends", is states:
ReplyDelete"Graphs should include clearly labeled error bars described in the figure legend. Authors must state whether a number that follows the ± sign is a standard error (SEM.) or a standard deviation (SD). The number of independent data points (N) represented in a graph must be indicated in the legend. Numerical axes on graphs should go to 0, except for log axes. Statistical analyses should be done on all available data and not just on data from a "representative experiment." Statistics and error bars should only be shown for independent experiments and not for replicates within a single experiment."
If you have an average and some calculated measure of dispersion, why not make a box plot? This simple chart gives some hint about the shape of the distribution, in addition to spread around the center.
ReplyDeleteThe Elements of Graphing Data by William S. Cleveland is an earlier reference and probably the source for your reference. Cleveland argues the using the standard error is a carryover from tables and does not make sense in graphs. The interval formed by standard errors give 68% confidence intervals which are not particularly interesting intervals. Providing the standard error in tables allows you to calculate the confidence interval of your choice.
ReplyDeleteJon: I‘m a big fan of box plots. But the whiskers can still be used to show different things - at least, I have the option to do that in my graphics software (Origin). So the problem remains that the identity of the whiskers has to be explicitly spelled out.
ReplyDeleteYoavram: The PNAS instructions provide an example of the differing practices in reporting statistical information. I've read some articles from statisticians that say SD or SE should never be preceded by ±, because you can’t have a negative SD or SE.