Wednesday, November 25, 2009

Forecast spread

I have been reading "The Black Swan" by Nassim Nicholas Taleb. It is written in a kind of arrogant style. I don't find it all that annoying perhaps because I largely agree with the book (at least so far). However it does encourage my natural inclination to nitpick.

On page 150-151 Taleb claims:

... Worse yet, the forecaster's errors were significantly larger than the average difference between individual forecasts, which indicates herding. Normally, forecasts should be as far from one another as they are from the predicted number. ...

This claim which I believe I have seen elsewhere is just not true. There is no reason to expect even independent unbiased predictions to behave in this way. Suppose for example we ask for forecasters to predict how many heads will be observed in 100 flips of a fair coin. Naturally they will all predict 50 but that doesn't mean we can expect exactly 50 heads. This is an artificial example but you will see the same clustering whenever there is a large random component in the variable being forecast. This variable could be for example the number of homicides in New York in 2010 or the size of the corn harvest in Iowa. Forecasters will be in effect estimating the mean of an underlying random distribution and there is no reason to expect these estimates to be as widely spread as the distribution itself.

No comments:

Post a Comment