The US military has been conducting a large and expensive AIDS vaccine trial in Thailand. Recently positive results were announced with much fanfare .
However the actual results are not that impressive. 74 out of about 8000 in the placebo group became infected as opposed to 51 out of about 8000 in the vaccine group. An approximate test of significance is to ask if we flip a fair coin 125 times what percentage of the time will the head tail split be at least as uneven as 74/51. By my calculation 4.87%. So this is statistically significant at the 5% level but just barely so. Again by my calculations if there had been one fewer infection in the placebo group this value would have been 5.89% or if there had been one more infection in the vaccine group this value would have been 6.09% neither of which would have been significant.
This illustrates one of the problems of picking an arbitrary line like 5% and declaring results on one side successes and on the other side failures. Another problem is the fact that there have been many vaccine trials other than this one all of which failed. Clearly if you run enough trials some will appear to succeed (at the 5% level) just by chance. I think there is a fair chance this is an example.