Statistical Bias4 分で読めます

The Average-of-Averages Error

If Class A averages 82 points and Class B averages 72 points, is the overall average 77 points? If the two classes have different numbers of students, this calculation is wrong.

Why the Average of Averages Is Wrong

Simply averaging the averages of differently sized groups again gives a wrong value. You must use a weighted average that reflects the size of each group.

For example, suppose Class A's 20 students average 82 points and Class B's 30 students average 72 points. Simply averaging the two averages gives (82+72)/2 = 77 points. But this is wrong.

The correct calculation weights by the number of people: (20×82 + 30×72) / 50 = 76 points. The overall average is pulled toward Class B, which has more students. Ignoring the difference in group size produces a value that differs from reality like this.

Simpson's Paradox, Which Even Flips the Direction

Ignoring group size does not just make the value slightly wrong; it can flip the direction of the conclusion itself. This is called Simpson's paradox.

It is the phenomenon where a certain trend appears within each group, but when the groups are combined, that trend disappears or appears reversed. The very reason this paradox arises is that each group has a different size and different conditions.

A Famous Case: UC Berkeley Admissions

In 1973, UC Berkeley's graduate school was suspected of discriminating against women. That is because, in the overall statistics, about 44% of male applicants were admitted, while only about 35% of women were admitted.

But when the data was broken down by department, the story changed. In most departments, women's admission rates were similar to or even higher than men's. How could this happen?

The secret was in the application patterns. Women applied more to departments with high competition (low admission rates). So a simple aggregation of the whole made women look disadvantaged, but once department size and competitiveness were accounted for, it was hard to call it discrimination. The researchers who did a stratified analysis concluded that it was, if anything, slightly favorable to women.

44% and 35% are widely cited approximate figures (Bickel et al., Science, 1975). The point is not the precise decimals but that "the overall average and the group-by-group averages pointed in different directions."

What to Watch for in Investing: The Geometric Mean

The same trap exists in investment returns. Simply taking the arithmetic average of yearly returns gives a value different from the compounded performance you actually pocket.

For example, if the first year was +50% and the next year was -50%, the arithmetic mean is 0%, but in reality $100 becomes $150 and then shrinks to $75, a -25% result. To properly represent compounded performance, you must use the geometric mean, not the arithmetic mean.

Behind the single word "average" hide several kinds like this. When this site shows returns, it correctly reflects the compounding effect and also displays the maximum drawdown and the loss period, so it does not hide the risk concealed behind the average.

よくある質問

Q. When must I use a weighted average?

Whenever the groups you are combining have different sizes (number of people, amount, weight, etc.), you must use a weighted average. Only when all groups are the same size do a simple average and a weighted average happen to coincide. If you use a simple average when the sizes differ, you get a value different from reality, and sometimes the conclusion can even flip.

Q. Why use the geometric mean rather than the arithmetic mean for returns?

Because returns have a compounding nature, multiplying and accumulating year after year. The arithmetic mean does not reflect this multiplicative effect and tends to be inflated above the actual compounded performance. To represent true multi-year performance, the geometric mean, which reflects multiplication, is correct.