Fact Checking

Recently New York Magazine published an amazing story about a Stuyvesant High School student who had purportedly  made $72 million in the stock market.  When I read it I was pretty sure it was nonsense as in fact it soon turned out to be.  As the magazine admits its fact checking was obviously inadequate.  I expect this was caused in part by a failure to realize just how unlikely this story was and the implications for the appropriate level of fact checking.   Suppose for example that there are a million false claims like this for every true account.  Then fact checking sufficient to catch 99.9% of the false claims is totally inadequate as 99.9% of the surviving claims will still be false.   As the saying goes "Extraordinary claims require extraordinary evidence".

New York Magazine stated:

... As part of the research process, the magazine sent a fact-checker to Stuyvesant, where Islam produced a document that appeared to be a Chase bank statement attesting to an eight-figure bank account. ...

But this is very ordinary fact checking.  Anybody can fake a bank statement.  The whole point of a bank reference is that you can verify it with the bank.  If New York Magazine had insisted on doing this they would likely have avoided this fiasco.