Good teaching, poor test scores

Evaluating teachers based partly on student test scores is unreliable, concludes a study in Educational Evaluation and Policy Analysis. Researchers analyzed a subsample of 327 fourth- and eighth-grade mathematics and English-language-arts teachers across six school districts.

“Some teachers who were well-regarded based on student surveys, classroom observances by principals and other indicators of quality had students who scored poorly on tests,” reports the Washington Post. Some poorly regarded teachers had students who did well.

Thirty-five states and the District of Columbia require student achievement to be a “significant” or the “most significant” factor in teacher evaluations. Just 10 states do not require student test scores to be used in teacher evaluations.

Most states are using “value-added models” — or VAMs — which are statistical algorithms designed to figure out how much teachers contribute to their students’ learning, holding constant factors such as demographics.

Last month, the American Statistical Association warned against used VAMS, saying that “recent studies have found that teachers account for a maximum of about 14 percent of a student’s test score.”

“We need to slow down or ease off completely for the stakes for teachers, at least in the first few years, so we can get a sense of what do these things measure, what does it mean,” said Morgan S. Polikoff, a USC assistant professor of education and co-author of the study. “We’re moving these systems forward way ahead of the science in terms of the quality of the measures.”

Houston eyes student grading of teachers

Houston may use student opinion as part of teacher evaluations, reports Fox 26. Student ratings could account for 20 to 30 percent of a teacher’s evaluation, according to a district document sent to teachers.

Effective teachers aren’t always the most popular teachers, warns Houston Federation of Teachers president Gayle Fallon. “Those student surveys will amount to very little more than a popularity contest.”

Student evaluation of teachers is not district policy — or even a staff recommendation — said district spokesman Jason Spencer. Not yet, anyhow.

Teacher observation: Imperfect, but a step forward

Evaluating teachers by watching them teach is “tricky, labor-intensive, potentially costly and subjective — but perhaps the best way to help them improve,” according to a Gates Foundation study (pdf) reported in the Los Angeles Times.

The findings highlight the importance of teacher observations, but also pinpoint why they frequently don’t work. The old way — observing a teacher once a year, or once every five years in some cases — is insufficient. And the observers, typically the school principal, frequently don’t know what to look for anyway.

But that doesn’t mean teacher observations should be tossed aside. The best way to evaluate teachers, while also helping them improve, is to use several measures — including data-based methods that rely on students’ standardized test scores, along with an updated teacher observation system, the report found.

Earlier research has looked at student surveys and value-added measures to judge teachers’ effect on students’ performance.

Using these methods to evaluate teachers is “more predictive and powerful in combination than anything we have used as a proxy in the past,” said Vicki Phillips, who directs the Gates project.

Traditionally, 98 percent of teachers are rated effective.

Researchers looked at “measures of success beyond test scores,” adds the Hechinger Report.

That is, can we know for sure that a teacher who receives a top grade on one of the more rigorous and frequent classroom observations is also going to have a classroom of students who get top grades on achievement tests at the end of the year and on other important measures, like interest and happiness in school? . . .  And are the evaluation measures, whether they are qualitative observations or quantitative test scores, accurate in labeling teachers great, ordinary, or bad?

Teachers’ observation scores correlated with their students’ results on a variety of achievement tests, the Gates study concluded.