The Signal and the Noise

I recently read "The Signal and the Noise" by Nate Silver.  Nate Silver became well known because of the political analysis and generally accurate election predictions in his blog 538.  Earlier he had developed a successful system, PECOTA, for predicting the development of minor league baseball players.  In this book he attempts a general survey of forecasting.  I found the book a bit disappointing (perhaps in part because I had excessively high expectations).

One problem with the book is that when Silver moves away from his specific areas of expertise he makes mistakes showing he has failed to completely grasp the material.  On page 110 Silver writes:

... They are hot: the 77 trillion calculations that the IBM Bluefire supercomputer makes every second generate a substantial amount of radiant energy.  They are windy: all that heat must be cooled, lest the nation's ability to forecast its weather be placed in jeopardy, and so a series of high-pressure fans blast oxygen on the computers at all times.  ...

Here "radiant energy" is at best a confusing way of saying heat.  And I had never heard of oxygen (as opposed to simple air) cooling.  In any case according to NCAR:

Bluefire relies on a unique, water-based cooling system that is 33 percent more energy efficient than traditional air-cooled systems. Heat is removed from the electronics by water-chilled copper plates mounted in direct contact with each POWER6 microprocessor chip. ...

The chess diagram on page 271 is obviously wrong (the pawn on g2 should be on g3) and the related statement on page 270:

These databases relied on the assumption, however, that Kasparov would respond as almost all other players had when faced with the position, by moving his knight back out of the way. ...

is simply false, the usual move is bg2, retreating the knight would be very weak.

On page 374 Silver writes:

The greenhouse effect is the process by which certain atmospheric gases - principally water vapor, carbon dioxide (CO2), methane and ozone - absorb solar energy that has been reflected from the earth's surface.  ...

Here "reflected" should be "absorbed by and re-radiated".  The key point is because the sun is hotter than the earth solar radiation has shorter wavelengths than the earth's heat radiation.  Greenhouse gases absorb more strongly at longer wavelengths hence block more of the outgoing radiation as compared to the incoming solar radiation.  The effect is to warm the earth's surface.  On the other hand sunshine reflected from the earth's surface remains shortwave and leaves as easily as it arrived.

These (and other similar) errors aren't critical to Silver's arguments and could be fixed without too much trouble but they diminish the reader's general confidence in the author's reliability.  Silver would have done better to have had subject matter experts check for this sort of thing. 

The book has other problems.  At 454 pages it is quite long and there are things like the chapter on the Kasparov - Deep Blue match which could have been cut without too much loss. 

I didn't care for the analysis of the failure of the rating agencies prior to the recent financial crisis.  Silver discusses technical issues but the real problem is the agencies are paid by the people selling the securities they are rating.  This gives them an obvious incentive to fudge their ratings.  Silver understands this when it comes to political polling, polls paid for by a candidate will generally report more favorable results for the candidate than neutral polls.  In the case of political polls the amount of fudging is limited by the existence of neutral and opposition polls.  There were few such countervailing forces when it came to rating complex mortgage backed securities and the rating agency fudging got totally out of hand.  At the end nearly worthless securities were being given AAA ratings.  Silver largely ignores the bad rating agency incentives which remain in place and seem likely to cause similar problems in the future.

I don't want to be too negative, the book contains some interesting material and I agree with a lot of Silver's conclusions.  But I can't really call it a must read.