10 May 2022

Reading bar graphs is harder than we thought

One of the most powerful ideas in statistics is that you can summarize an indefinite amount of data with just a couple of numbers. For example, a mean and standard deviation can tell you a lot about millions or data points.

For that reason, I’ve always had mixed feelings about the saying, “Friends don’t let friends use bar graphs.” The argument is that you should always plot all the data points because summary statistics can be misleading if the data have unusual distributions. This seemed to me to be throwing away one of the central ideas of statistics. In the Better Posters book, I argued that showing a summary bar graph is fine if the underlying data meet normal assumptions.

I am having second thought on that because of a paper by Kerns and Wilmer (2021) about how people read bar graphs.

Bar graphs can show at least two things: quantities or averages. Kerns and Wilmer are concerned with averages. They show that about one in five people (20%) misunderstand what a bar graph is showing. They treat the end of the bar as the upper limit of a range, not a mean.

Kerns and Wilmer call this the bar-tip limit error.

They found this by showing people bar graphs and asking them to place imaginary data points on the graph.

Two rows of bar graphs drawn by subjects. Top row: Data points are centered around bar tip. Bottom row: Data points are between X axis and bar tip.

“But,” you might say, “is this just a mistake people make because they don’t understand averages?”

Nope.

Graph of percentage of readouts that made the bar tip limit error. “All”: full dataset. “Correctly defined average/mean”: participants who correctly defined the mean. “Correctly identified average/mean”: participants who correctly identified a mean value on the same graph that produced the readout. “Correctly defined & identified average/mean”: participants who both correctly defined the mean and correctly identified a mean value on the graph. In all cases, the proportion of bar tip limit errors is about one in five. Vertical lines show 95% CIs, and gray regions show full probability distributions for the uncertainty around the percentage values.

Kerns and Wilmer show that even when people can define what an average is (second from left in graph above)...

And even when people can identify the average on the graph (third from left above)...

And even when people can do both (rightmost in graph above)...

About 20% of people still make the mistake of placing all the data points below the bar tip.

“But,” you might say, “maybe people with more advanced training would not make this mistake? At an academic conference, most of the people are in or have gone to graduate school.”

Nope – well, mostly nope. General education alone doesn’t seem to make much difference, but specific education might.

Scatterplot. X axis: Education level, grouped into high school, some university education, university education, and graduate education. Mean bar tip limit measures for each participant plotted against general education level, where high values indicate subject made bar tip limit error. Black line: least-squares regression line. All four categories show a persistent minority make the bar tip limit error.


In the graph above, high values near 100 show a subject made bar-tip limit error. All four groups show a persistent minority – again, about one in five – make the bar-tip limit error.

But lest you think this is some sort of “hard wired” cognitive bias for some people, mistakes do seem to decline as people received more formal training in statistics.

Scatterplot. X axis: Statistics training, grouped into none, less than 1 course, one course, and more than one course. Y axis: Mean bar tip limit measures for each participant plotted against general education level, where high values indicate subject made bar tip limit error. Black line: least-squares regression line. All four categories show some individuals make the bar tip limit error, but fewer make the error if they have more statistics courses.

Nevertheless, even with multiple courses in statistics, some people continue to make the bar-tip limit error.

This might explain why there are so many people who continually complain about “Half the country is below average!” Hat tip to Darach Ó Séaghdha on Twitter for spotting this example:

Headline rading, "Several counties still have a higher virus incudence rate than the national average"

It is worth creating bar graphs with the knowledge that some people won’t read the graph correctly. Twenty percent is a lot of viewers. For comparison, think about the attention designing for colour blindness gets, even though probably less than 5% of people at a typical conference are colour blind.

On a poster, I worry about the level of visual complexity inherent in graphs where the major point is to show differences in averages. I say again: summarizing data is powerful. But I see the value in helping readers avoid falling into the bar-tip limit error.

I suggest that your bar graphs of averages use colour to emphasize the bar, so that it is visible at a distance, and use low contrast to show the individual data points. (See this post on putting make-up on your graphs.)

This is the scenario tested by Kerns and Wilmer:

Generic bar graph wth three bars.

As far as I can tell, Kerns and Wilmer did not test the effect of adding any sort of error bars to the graph, like standard deviation or confidence interval. So maybe that would help avoid the error:

Generic bar graph wth three bars with error bars.

But maybe the best way to break the bar-tip limit error is just to add the data.

Generic bar graph wth three bars with error bars and data points.

But this is now getting visually complex. The colours used to fill the bar above are a little intense. Large areas of colour still read as a colour, so those are a bit more subdued.

Generic bar graph wth three bars with error bars and data points, but data points are low contrast..

The data points are a neutral – light grey – and have a 50% transparency. The data are visible, but hopefully not too distracting from a distance. This particular graph above may not be perfect in terms of the right balance of contrast and colours, but indicated what I think is a good approach.

References

Kerns SH, Wilmer JB. 2021. Two graphs walk into a bar: Readout-based measurement reveals the Bar-Tip Limit error, a common, categorical misinterpretation of mean bar graphs. Journal of Vision 21(12): 17. https://doi.org/10.1167/jov.21.12.17

Related posts

Never use a graph you can’t explain

Lessons from make-up

External links

Can you read a bar graph? (NPR Science Friday)

1 comment:

  1. I always thought that bar plots are misleading because, even when they refer to a normally distributed measurement, they are an asymmetric representation of a symmetric distribution - this is intrinsically problematic.

    ReplyDelete

Comments are moderated. Real names and pseudonyms are welcome. Anonymous comments are not and will be removed.