Bland-Altman Plot
Bland-Altman Plot. A chart for judging whether two measurement methods agree: each pair's difference is plotted against the pair's average, with limits of agreement.
Agreement is not the same as correlation
Suppose you log twenty meals in an app and also know each meal’s true calories from weighing. You could correlate the two columns, and you would almost certainly get a high correlation, because big meals are big in both columns. That says very little. Two methods can correlate at 0.95 and still disagree by a hundred calories on every plate.
In a widely cited 1986 paper in The Lancet, the statisticians Martin Bland and Douglas Altman set out a better question for this situation: how far apart are the two methods, and does that gap change with the size of the thing being measured? Their answer was a simple chart that has since become a standard tool in clinical measurement.
How the plot is built
For every pair of measurements, here the app’s value and the true value:
- Work out the difference, app minus truth.
- Work out the average of the two.
- Plot the difference on the vertical axis against the average on the horizontal axis.
- Draw a line at the mean difference. This is the bias: how much the app over- or under-counts on average.
- Draw two more lines at the mean difference ±1.96 standard deviations of the differences. These are the limits of agreement, the range within which about 95% of differences are expected to fall.
Worked example
Five meals, with true calories from weighed components and the app’s logged value:
| Meal | True | App | Difference | Average |
|---|---|---|---|---|
| 1 | 400 | 420 | +20 | 410 |
| 2 | 550 | 540 | −10 | 545 |
| 3 | 700 | 735 | +35 | 717.5 |
| 4 | 300 | 310 | +10 | 305 |
| 5 | 850 | 880 | +30 | 865 |
The mean difference is +17 kcal, so this app over-counts slightly on average. The standard deviation of the differences is about 17.9 kcal, so the limits of agreement are 17 ± 1.96 × 17.9, roughly −18 to +52 kcal. In plain terms: for a meal like these, the app will usually land between 18 kcal under and 52 kcal over the truth.
Five pairs are far too few for real conclusions. With so small a sample the limits are themselves very uncertain, which is why proper studies use dozens or hundreds of meals and report confidence intervals for the limits as well as the bias.
Reading the shape
The plot earns its keep when you look at the pattern of the points, not only the three lines:
- A flat band offset from zero. A fixed bias: the app misses by about the same number of calories whatever the meal size.
- A band that slopes. The difference grows or shrinks with meal size, so the error is proportional rather than fixed.
- A funnel that widens to the right. Small meals agree closely, large ones scatter, so a single pair of limits understates the error on big plates.
A sloping or funnel-shaped plot is a signal to report error as a percentage, or to report limits separately for small and large meals, rather than quoting one figure in calories.
Where it fits among the other metrics
MAPE and MAE summarise how big the typical miss is. A Bland-Altman analysis adds two things they do not: the direction of the bias and whether the error depends on meal size. They are complements, not rivals, and a validation report that includes both tells a reader far more than either alone. Our article on MAPE vs MAE vs MAD shows how the summary metrics can rank the same apps differently; a Bland-Altman plot is often the quickest way to see why.