Last week Kevin Drum posted about a paper (pdf file, may be behind a paywall) by Eric Hanushek which estimates that a teacher one standard deviation above mean will improve the present value of a typical student's lifetime earnings by a substantial amount ($20000 with one set of assumptions). If correct these estimates would justify considerable effort and expense improving the average quality of teachers. However as readers of this blog know I doubt teachers make much difference. This paper has not convinced me otherwise.
The paper's estimates depend on a number of extremely debatable assumptions. Even worse it appears to contain a blatant error. One reason teachers don't matter much in the long run is the measured effects they do have tend to be temporary, fading away with time. Hanushek recognizes this and defines (p 14-15) a depreciation variable theta. But he does not appear to realize the effects are cumulative. After n years the effects of a particular teacher (as measured by achievement tests) are reduced by a factor of (1-theta)**n. But Hanushek assumes (1-theta) of the effect is permanent. Since theta is estimated as .3-.6 this makes a huge difference in his calculations. So, even granting the assumptions in his model, correctly computed the effect of an above (or below) average teacher will be much less than claimed.
The cumulative nature of this effect is clearly stated in one of the papers Hanushek cites Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation ((c) 2008 Kane and Staiger, p.4):
Finally, in the experimental data we found that the impact of the randomly assigned teacher on math and reading achievement faded out at a rate of roughly 50 percent per year in future academic years. In other words, only 50 percent of the teacher effect from year t was discernible in year t+1 and 25 percent was discernible in year t+2. ...
Raw data: A cautionary tale
7 hours ago
..depends on how you measure success.. True, as time passes, there will be less of an impact, however, some things are still effective....how do you account for the fact, for example, that Mrs. McGlinchly, your third grade teacher, still have as effect on how you sign a personal letter?
ReplyDeleteThe mediocre teacher tells. The good teacher explains. The superior teacher demonstrates. The great teacher inspires.
ReplyDelete- William Arthur Ward
1) Building on / supporting Mary's comment - there is a lot of value that education provides beyond that which can be measured economically. The broadening and deepening of one's perspective that education facilitates can enrich life greatly beyond what can be measured in financial terms.
ReplyDelete2) I can't access Kane/Staiger but I'm wondering whether they *assumed* exponential decay. It seems implausible to me that the annual decay rate of the effect of an excellent Kindergarten teacher would be as great during 12th grade as it was during 1st grade.
Hanushek uses a model which in effect assumes exponential decay but does not appear to realize this. See my follow up post for more details.
ReplyDeleteKane and Staiger measured decay consistent with exponential decay for the first couple of years. I suspect the effects for additional years become indistinguishable from noise. My earlier post on teachers contains another link to the Kane and Staiger paper which you may be able to access.