I recently blogged about Steven Brill's rubber room article in the New Yorker. Brill mentions in passing a study that purportedly showed that there are important differences in teacher effectiveness.
... A study of the Los Angeles public schools published in 2006 by the Brookings Institution concluded that “having a top-quartile teacher rather than a bottom-quartile teacher four years in a row would be enough to close the black-white test score gap.” ...
The study in question appears to be Identifying Effective Teachers Using Performance on the Job (pdf file) by Gordon, Kane and Staiger.
Briefly the authors construct a model for predicting student performance on standardized assessment tests at the end of the school year based on their performance on similar tests the previous year and some other factors such as sex, race, English language ability etc. The models implicitly assume the students will be taught by an average teacher. They then compare the actual results with the predicted results and assert the difference averaged over the students in a class reflects in substantial part differences in teacher effectiveness (relative to an average teacher). In particular teachers ranked high based on the first two years of their teaching career will perform noticeably better (compared to teachers ranked low) in their third year of teaching (based on how well their classes test at the end of the year relative to the results predicted for an average teacher).
The authors go on to make some policy recommendations. They note that although teachers are generally subject to a probationary period of 2-3 years at the beginning of their career with a school district before they are given tenure (and become almost impossible to fire) it is rare for a teacher to be denied tenure. They recommend ranking probationary teachers based in part on their methods and presumptively denying tenure to candidates in the bottom quartile. They make some rather extravagant predictions of social benefits from adopting this policy nationally based on some rather heroic assumptions.
An obvious question about these results is how long the improved student performance lasts. It is fairly common for short term gains to disappear in the long run. Unfortunately according to a later paper, Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation (pdf file) by Kane and Steiger the effects quickly dissipate.
... In both the experimental and non-experimental data, we found that teacher effects faded out by roughly 50 percent per year in the two years following teacher assignment.
So I guess I may need to modify my earlier claim that "teachers don't matter much" to "teachers don't matter much in the long run".
I have some other criticisms of the earlier paper but if the gains don't persist they aren't very important even if they are real.