Misleading graphs
Misleading graphs are charts whose numbers are correct and whose picture is not, usually because the axis, the scale or the range has been chosen to make a difference look larger or smaller than it is. Six devices do most of the work: the truncated axis, the dual axis, area used to represent length, the cherry-picked date range, the uneven interval and the three-dimensional effect. Each has a visual signature, and each can be caught in about ten seconds once you know where to look.
None of the six requires a false number. That is what makes them durable. A chart that lies about a value can be corrected by anybody with the underlying data; a chart that draws a true value at a misleading size passes every check except the one nobody runs, which is reading the axis before reading the shape.
What misleading graphs are
A misleading graph is a visualization in which the geometry and the data disagree. The eye reads charts by comparing lengths, areas and slopes, and it does that comparison before conscious reading begins. A chart designer who changes what a unit of length is worth, without the reader noticing, has changed the conclusion without touching a number.
What distinguishes a misleading graph from a bad one is intent-free consistency: the distortion always runs in the direction the presenter wants. A genuinely careless chart makes errors in both directions. A chart in which every choice, the axis start, the range, the color emphasis and the scaling, happens to favor one reading is not careless. In a set of six charts in a deck, that pattern is worth more than any single chart in it.
The one-sentence version: read the axis before you read the shape
In one sentence: look at the numbers on the vertical axis, the units on the horizontal axis and the range of dates covered, and only then look at the picture. That order of operations defeats most misleading graphs on its own.
Three signatures deserve immediate suspicion. A bar chart whose bars do not start at zero, since bar length is the entire message of a bar chart. A chart with two vertical axes, since the point where the two lines cross was chosen by whoever set the scales. A chart whose date range begins or ends at an odd point, such as a series that starts in March or ends in a month that is not the most recent available.
The arithmetic worked through: what a truncated axis does to a 4 percent gap
Work the arithmetic on two bars representing 92 and 96, drawn on a vertical axis that runs from 90 to 100 instead of from 0. The values are true. The difference between them is 4 units, which is 4.3 percent of 92.
- On a zero-based axis: the bars stand 92 and 96 units tall, and the second is 4 percent taller. The picture matches the arithmetic.
- On an axis starting at 90: the bars stand 2 units and 6 units tall, because only the part above 90 is drawn.
- The visual ratio becomes 6 divided by 2, which is 3. The second bar carries three times the ink for a value 4.3 percent higher.
- Push the axis start to 91 and the bars become 1 and 5 units, a visual ratio of 5 to 1 for the same 4.3 percent gap.
The distortion has no upper limit, because the truncation point is a free parameter. Move the baseline close enough to the smaller value and any difference can be made to look like any multiple you please. That is why the baseline is the first thing to check and why the size of the distortion cannot be judged from the picture: it is a function of a number that is printed in small type at the bottom left.
Area works the same way and is worse, because the exaggeration is squared. Represent a quantity that has risen by 50 percent using an icon scaled to 1.5 times its width and 1.5 times its height, and the icon's area becomes 1.5 times 1.5, which is 2.25 times the original. Represent a doubling with a circle of double the radius and the area quadruples, since area grows with the square of the radius. The reader sees ink, the caption reports a length, and the gap between them is a factor of two or more.
What a failure looks like: six devices and their signatures
A misleading graph failure is easiest to recognize as a catalog, because each device leaves a specific mark on the chart. The examples below are constructed rather than collected, so that every number in them can be checked against the description.
| Device | What the picture does | Where to look | The fix |
|---|---|---|---|
| Truncated axis | Turns a 4 percent gap into a threefold visual gap | The lowest label on the vertical axis | Redraw from zero, or use a dot plot with the values printed |
| Dual axes | Makes two unrelated series appear to track or diverge | Two sets of numbers on left and right | Split into two charts, or index both series to 100 at the same date |
| Area for length | Squares the difference, so 1.5 times looks like 2.25 times | Icons, circles or pictures scaled in both dimensions | Scale area to the value, or use plain bars |
| Cherry-picked range | Reverses the direction of a trend | The first and last date on the horizontal axis | Show the full available series and mark the excerpt |
| Uneven intervals | Stretches or compresses time to change the slope | Gaps between horizontal axis labels | Space points by actual elapsed time |
| Three-dimensional effect | Enlarges whichever element is nearest the viewer | Perspective on pie slices or bar tops | Remove the perspective entirely |
The cherry-picked range deserves its own worked figure, since it is the only one that can change a sign. Take an invented monthly series that reads 250 in January, peaks at 310 in April and finishes the year at 288. Charted from January the series is up 15.2 percent. Charted from April it is down 7.1 percent. Both charts are accurate, neither omits a data point within its own window, and they support opposite headlines.
The common misreading: assuming a truncated axis is always dishonest
The common misreading is that any chart not starting at zero is deceptive. That rule is too strong and following it produces its own bad charts. For a line chart of a quantity that varies within a narrow band, forcing the axis to zero flattens every real movement into a straight line and hides the signal. For quantities with no meaningful zero at all, such as a temperature in Fahrenheit or a calendar year, a zero baseline is meaningless.
The usable rule is about the encoding rather than the number. Bars encode value as length, so a bar chart must start at zero or the length lies. Lines encode value as position and change as slope, so a line chart may start anywhere the axis is clearly labeled. A second misreading is the belief that a chart with all its data points printed cannot mislead: a scatter of correct points on a distorted axis still misleads, because the picture is read first and the labels second.
The test to run on any chart
Run these five checks on a chart before you form an opinion about what it shows.
- Read the vertical axis first, and note the lowest value printed on it.
- Count the axes, and treat any chart with two vertical scales as making no claim about the relationship between its lines.
- Check the endpoints of the horizontal axis, and ask what the series did just before and just after the window.
- Measure two elements you can compare, and check whether their ink ratio matches their value ratio.
- Ask what the same data would look like drawn the plainest possible way.
The fifth question is the one that settles arguments. If a chart drawn as a plain zero-based bar chart or a plain line at full range shows something noticeably weaker than the original, the difference between the two pictures is the work the design was doing. Reading numbers covers the same skepticism applied to figures in prose, The null hypothesis supplies the question of whether a visible gap would have appeared by chance anyway, and The gambler's fallacy is what a run of points in a chart tempts a reader into.
Finding real examples, and fixing them
Real misleading graphs turn up in four places reliably, and collecting them is the fastest way to learn the signatures, which is why this is a standard exercise for students. Look in quarterly earnings presentations, where truncated axes on growth charts are near universal. Look in product comparison slides, where a competitor's figure is drawn against a scale chosen for the sponsor's figure. Look in news graphics under deadline, where dual axes and short date ranges appear because the full series did not fit. Look in online forums that collect charts specifically for their errors, keeping in mind that a chart posted as an outrage is sometimes correct and merely unflattering.
Fixing one is the exercise that teaches most. Take any chart you suspect, write down the values it encodes, and redraw it with three constraints: a zero baseline for anything drawn as a bar, one vertical axis, and the longest date range the source actually publishes. Then compare the two pictures side by side. Whatever difference remains between them is the size of the claim the original was making with geometry instead of with data, and putting a number on that difference is the whole skill.
There is no misleading graph calculator, but there is a number you can compute
No calculator detects a misleading graph, because the distortion is a relationship between a picture and a dataset, and a tool that could see both would already have the honest chart. What you can compute by hand is the lie factor, defined by Edward Tufte in The Visual Display of Quantitative Information in 1983 as the size of the effect shown in the graphic divided by the size of the effect in the data. Tufte's standard is that the ratio should sit close to 1.
Run it on the two examples worked earlier on this page.
- The truncated bars: the visual ratio is 6 units to 2 units, which is 3.0. The data ratio is 96 to 92, which is 1.043. The lie factor is 3.0 divided by 1.043, or about 2.9, meaning the picture overstates the difference by nearly threefold.
- The icon scaled in two dimensions: the visual ratio is 2.25 to 1 and the data ratio is 1.5 to 1, so the lie factor is 1.5. The exaggeration is smaller and it is built into the geometry rather than into the axis.
Two things have to be settled before the lie factor can be computed, and both are outside the chart. You need the underlying values, which a distorted chart often does not print, and you need to decide which two elements are being compared, since a chart with eight bars has twenty-eight possible pairs and they do not all carry the same lie factor. Measuring the ink is the easy half. Obtaining the data the ink is supposed to represent is the half that fails.
What the lie factor does not settle is whether the chart is honest, and this is the limit worth carrying away. It measures one thing precisely: whether the geometry matches the arithmetic. A chart can score exactly 1.0 and still mislead completely, because the cherry-picked range, the missing comparison and the omitted series all distort by leaving true data out rather than by drawing the remaining data at the wrong size. The January to April series described above has a lie factor of 1.0 in both versions and supports opposite headlines. A number tells you the picture is drawn to scale. Whether it is drawn of the right thing is a question no measurement of the image can reach.