On ranking teachers

The Los Angeles Times ranked teachers by value-added scores based on “demonstrably inadequate” research, concludes a National Education Policy Center study, Due Diligence and the Evaluation of Teachers. University of Colorado researchers found “serious weaknesses” in the LA Times’ work.

. . .  it is likely that there are a significant number of false positives (teachers rated as effective who are really average), and false negatives (teachers rated as ineffective who are really average) in the L.A. Times’s rating system. Using the Times’s approach of including only teachers with 60 or more students, there was likely a misclassification of 22% (for reading) and 14% (for math).

But that’s not how the newspaper sees it. Separate study confirms many Los Angeles Times findings on teacher effectiveness is the Times’ Feb. 7 headline, claiming NEPC  “confirms the broad conclusions of a Times’ analysis of teacher effectiveness in the Los Angeles Unified School District while raising concerns about the precision of the ratings.”

After re-analyzing the data using a somewhat different method, the Colorado researchers reached a similar general conclusion: Elementary school teachers vary widely in their ability to raise student scores on standardized tests, and that variation can be reliably estimated.

But they also said they found evidence of imprecision in the Times analysis that could lead to the misclassification of some teachers, especially among those whose performance was about average for the district.

NEPC is livid about the Feb. 7 story (pdf), saying the study by Derek Briggs and Ben Domingue “confirms very few of the Times’ conclusions.”

“We raised major concerns was with both the validity (“accuracy”) and reliability (“precision”), and our bigger focus was on the former rather than the latter,” writes Professor Briggs.

While the Times claims the study “largely confirmed” the newspaper’s rankings of most and least effective teachers, Briggs responds, “No, we did not, quite to the contrary. (Reporter Jason) Felch seems to be again focused only on the
precision issue and not on the accuracy problems that we primarily focus on in our report.”

About Joanne

Comments

  1. All I’ve got is one word: terrifying!

  2. My favorite quote from the Times;

    The authors also found that the way school administrators assign students to teachers — giving especially challenging students to a certain teacher and not to others, for example — could have skewed the value-added results.

    …file that under, no duh.

  3. Peace Corps says:

    I say take the top 5% of the value added scores and reward them. Look at the bottom 5% and see what is going wrong in those classrooms. I wouldn’t ever just look at the value added information, but the teachers that score at the very top are probably doing a lot of things right and the teachers that score at the very bottom are probably not doing very much right.

  4. @Peace Corps
    or those top 5% never get special ed students or any of the “bad” students and the bottom 5% do.

    I have been told numerous times that our special ed department won’t even bother putting a student into certain teachers’ classrooms because they won’t get the “extra attention” needed.