I recently blogged about Steven Brill's rubber room article in the New Yorker. Brill mentions in passing a study that purportedly showed that there are important differences in teacher effectiveness.
... A study of the Los Angeles public schools published in 2006 by the Brookings Institution concluded that “having a top-quartile teacher rather than a bottom-quartile teacher four years in a row would be enough to close the black-white test score gap.” ...
The study in question appears to be Identifying Effective Teachers Using Performance on the Job (pdf file) by Gordon, Kane and Staiger.
Briefly the authors construct a model for predicting student performance on standardized assessment tests at the end of the school year based on their performance on similar tests the previous year and some other factors such as sex, race, English language ability etc. The models implicitly assume the students will be taught by an average teacher. They then compare the actual results with the predicted results and assert the difference averaged over the students in a class reflects in substantial part differences in teacher effectiveness (relative to an average teacher). In particular teachers ranked high based on the first two years of their teaching career will perform noticeably better (compared to teachers ranked low) in their third year of teaching (based on how well their classes test at the end of the year relative to the results predicted for an average teacher).
The authors go on to make some policy recommendations. They note that although teachers are generally subject to a probationary period of 2-3 years at the beginning of their career with a school district before they are given tenure (and become almost impossible to fire) it is rare for a teacher to be denied tenure. They recommend ranking probationary teachers based in part on their methods and presumptively denying tenure to candidates in the bottom quartile. They make some rather extravagant predictions of social benefits from adopting this policy nationally based on some rather heroic assumptions.
An obvious question about these results is how long the improved student performance lasts. It is fairly common for short term gains to disappear in the long run. Unfortunately according to a later paper, Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation (pdf file) by Kane and Steiger the effects quickly dissipate.
... In both the experimental and non-experimental data, we found that teacher effects faded out by roughly 50 percent per year in the two years following teacher assignment.
So I guess I may need to modify my earlier claim that "teachers don't matter much" to "teachers don't matter much in the long run".
I have some other criticisms of the earlier paper but if the gains don't persist they aren't very important even if they are real.
Raw data: A cautionary tale
7 hours ago
Thanks for the links - they make indeed an informative reading. I still disagree with your conclusions, but do appreciate that you relate them closely to these outcomes. I consider this question from a rather high-level point of view: it simply cannot be a valid conclusion as it would contradict our long-standing experience. It is easy to find numerous examples of brilliant teachers producing brilliant students. Teacher may not be able to "produce" talent in students but talent certainly needs a great teacher to support its development. I am not sure how such a thing can be measured. The authors (in the second study) themselves seem to be cautious about drawing conclusions from the apparent fading effect, which could be due to the fact that old knowledge is not tested in the relevant year's standardized tests, or due to overall class improvement (due to the teacher).
ReplyDeleteTeachers matter, and that is probably symmetrically true: just as they can excite student's mind to pursue new directions, they can fail him/her, or, even worse, discourage from pursuing what might become their life-long passion.
I like the idea that teachers can indeed discourage students from following their passion...Our teachers were overwhelmed with double sessions and tons of students in the post war baby boom...they didn't seem to time time for anything except grading workbooks.
ReplyDeleteSmall changes over time do make a difference. Add this study to other ideas and programs and results can be made. Early infant stimulation for example may help (most brain development and early stimulation from parents occurs in the first three years). Also a push to teach English to 2nd language learners woyld help.Another thing that would help would be to recruit really imtelligent Math and Science people....they don't want to teach as te salary is not competetive with other professions. Instead, everyone takes pot shots at education instead of going to work to find out what really works.
There are currently about 50 million students and 3 million teachers in US K-12 public schools. Few of them are brilliant. What I am saying is for students and teachers in the broad middle range of ability differences in performance appear to be far more dependent on differences among students than differences among teachers.
ReplyDeleteI understood correctly what you were saying the first time, and that indeed may be close to what the results of the study are also indicating - agreed. The danger is in the statement "teachers don't matter" standing alone without context.
ReplyDeleteThe study might just be telling us that the variance in teachers quality in LA is rel. small and hence, on average, it doesn't matter much whether you get teacher A, and not teacher B, in that context.
I knew a teachers once who taught in Watts. The neighborhood was so bad that the school system bussed the teachers into the schools due to the danger to teachers in the neighborhoods. One can only imagine what these teachers went through once they got into a class. I believe that mostly they must have tried to keep order, period, if possible. Educating the students was a small part of the total picture. I agree that neither teacher Anor teacher B would make much of a difference in that situation. You could study such a school population until the cows ome home. It wouldn't make any difference.
ReplyDeleteThis is an area where statistics do not tell the story of individuals benefiting from good teachers. As in my other comments (Seniority pay, 9/9/09, and Rubber Rooms 8/27/09), I am basing my understanding of the teacher's role on discussions I have had with a middle school teacher in a suburban community in the San Francisco Bay Area.
ReplyDeleteMy teacher friend told me of a young lady who recently approached her in the Mall. Calling her by name, she said that my teacher friend might not remember her, but she wanted to tell her what a positive influence she had made on her life. She is now studying for a BA in political science at San Jose State University because my friend had made political science seem so interesting when she taught the course.
When my friend first started teaching, it was in a Home Economics class in the high school. She was appalled to find there was no order in the classroom. Students were talking all the time and throwing things across the room. She asked the principal for a day off so that she could visit other classes in the area. She went to Home Ec. classes in Pleasanton, a nearby community, and saw that their students were well behaved. So she determined this was the way her classes would be. Now all her classes are orderly, and the students are well behaved, which, she pointed out, is important if the students are going to be able to focus on learning and are motivated to learn. Incidentally my friend is well liked by her students; her only complaint is that if another teacher is having trouble with a student,
the principal would like to transfer the student to my friend's class, and there is a limit on how many of these students can be absorbed successfully into a well functioning classroom during one school year.
She believes that improved reading skills are never lost. If a good teacher can bring a student up from such a low reading level that reading is painful to a level where it is easy and reading is pleasant, this skill will never be lost. She has her success stories: There was the little retarded girl who entered my friend's 8th grade class only able to write her name and read at a first grade level. The child's mother was apprehensive about her daughter being put in my friend's class because of the pressure that might be put on her, but agreed to try it. At the end of the school year, the little girl was reading the Laura Ingalls Wilder books. My friend saw the mother during summer and was told that her daughter was still reading and enjoying it.
In another case, a mother agreed to have her daughter repeat 8th grade with my friend. At the end of the second year, she had advanced from a 6th grade reading level to a 9th grade level and was ready to go into high school.
Good teachers make a long term difference in the lives of some of their students. No doubt there are some students who are proficient in learning on their own, but for others a good teacher changes their lives.
It's great to hear such tales of success.
ReplyDelete(My favorite tale involved a student I had had in elementary School who came back after a year in jail because he had received all A's in the jail high school. There was no one else that was in his life that he felt could validate his success except for me.
Yes, talented, experienced teachers such are you mention can help students advance. The problem is that teachers may see the changes, but statisticians slice and dice the stats to show that we have made little progess. These same individuals should be using their talents in a classroom. It may help them to see what problems we are dealing with, and possibly to help some struggling student as well. Teaching is not like canning tomatoes. On a factory line the workers get to throw out the rotten ones, the wierd shaped ones, or the ones with nicks in them. Our mandate as teachers is to help every single child. We are required to teach all, for better or worse.