Merit pay fails a test

Is merit pay a flop? Project on Incentives in Teaching (POINT) offered big bonuses to Nashville math teachers in grades 5-8 for raising students’ test scores. Fifth graders improved in the second and third years, but there was no lasting improvement in student performance, concluded researchers at Vanderbilt’s National Center on Performance Incentives.

The study used a control group and calculated students’ progress using value-added measures. There was heavy attrition as teachers were reassigned or left the district.

In surveys of participants, 80 percent said they didn’t change their teaching in hopes of earning a bonus, according to the Hechinger Report. Teachers’ students had to hit the 95th percentile to earn a $15,000 bonus, the 90th percentile for $10,000 and the 80th for a $5,000 reward. Many teachers just missed the cut-off.

The fact that many fifth-grade teachers teach multiple subjects to the same students may have been a reason for the positive impact of merit pay found in fifth grade, according to the study’s authors. But “the effect did not last. By the end of 6th grade it did not matter whether a student’s 5th grade math teacher had been in the treatment or control group,” the study said.

While “bonus pay alone” didn’t improve student outcomes, “more nuanced” compensation ideas should be tested, said Matthew Springer, executive director of the National Center on Performance Incentives.

Merit pay opponents say the study proves merit pay is worthless. Others claim the real issue is not whether bonuses motivate teachers to work harder.

Under Arne Duncan, the Education Department is pushing performance pay, writes Stephen Sawchuk on Teacher Beat. New grantees will be announced this month “under a federal program designed to seed merit-pay programs for teachers and principals.”

“While this is a good study, it only looked at the narrow question of whether more pay motivates teachers to try harder,” a spokeswoman for the U.S. Department of Education said in an e-mail. “What we are trying to do is change the culture of teaching by giving all educators the feedback they need to get better while rewarding and incentivizing the best to teach in high need schools and hard-to-staff subjects.”

Before the results were out, Rick Hess wrote that it would tell us nothing because it looked only at whether teachers will work “harder” for money, like rats trying to earn extra food pellets. “Serious people” hope that “rethinking teacher pay can help us reshape the profession to make it more attractive to talented candidates, more adept at using specialization, more rewarding for accomplished professionals, and a better fit for the twenty-first century labor force.” The study asked the wrong question, he concludes.

Eduwonk agrees that merit pay is a way to improve the teaching force of the future, not a way to make teachers work harder or better.

It sends a signal that in this field, performance and excellence matters.  Right now the signal is that everyone gets treated alike, as widgets, regardless of how well or how poorly you do your job.

A “surprising number of people are saying this somehow settles the debate about performance pay,” Eduwonk also notes.  “And funny, they don’t say that about single studies that don’t confirm their views.”

However, performance pay should be judged on performance, argues Intercepts, who links to the response from the AFT, which sees a “a role” for performance pay,  and from the NEA, which calls it “only the latest blow” to the idea. Intercepts writes:

It’s funny to see NEA suddenly equating “student achievement” with the results of a “single standardized test” (page 13) – in this case, the Tennessee Comprehensive Assessment Program (TCAP) math test.

The Tennessee results aren’t all rosy for the teachers’ unions . . . if your goal is to raise student math scores, and a $15,000 bonus to math teachers didn’t do it, why would giving all teachers more money have any effect?

“What happens next is just as much a political question as an education one,” concludes Intercepts. Very true.

Update: Rick Hess publishes a response to the study by Tom Kane, a Harvard professor who’s heading the Gates Foundation’s  research into teacher performance, evaluation, and pay. Kane writes:

It’s a well-done study of a not-very-interesting question. Merit pay for teachers could impact student achievement via three distinct routes: by encouraging teachers to work harder, by encouraging talented and skilled teachers to remain in teaching, by enticing talented and skilled people to enter teaching. The study was designed to answer a narrow question: can you make the average teacher work harder with monetary incentives? They did not report any results on the likelihood that more effective teachers would remain in teaching. Nor did they design the study to study entry into teaching.

We know there are huge differences in student achievement gains in different teachers’ classrooms. The authors confirmed that result. However, the impact of the specific incentive they tested depends on what underlies the differences in teacher effectiveness–effort vs. talent and accumulated skill. I’ve never believed that lack of teacher effort–as opposed to talent and skills–was the primary issue underlying poor student achievement gains. Rather, the primary hope for merit pay is that it will encourage talented teachers to remain in the classroom or to enter teaching.

Kane thinks “more meaningful tenure review” is “the most likely route of impact for teacher effectiveness policies.”

By the way, Corey Bunje Bower, a Vanderbilt PhD student, writes that a “reliable source” says Hess knew the results of the study before he wrote the pre-announcement column saying the results don’t matter. I’m reluctant to take the word of an anonymous source over the word of Rick Hess.

Hess says someone told him the results after he’d written the column.


About Joanne

Comments

  1. Homeschooling Granny says:

    If we are concerned about the quality of teachers, shouldn’t we question whether the ed schools that prepare and certify them are doing the best job possible. If we were to identify all the poor teachers and encourage them to seek employment elsewhere we would then need to hire more teachers–and they would be prepared by the very same ed schools. If we need better teacher, don’t we need better ed schools?

  2. The union embraced the study design. That made me wonder. Why? Seemed like a positive finding would create problems for them.

    Then I read the actual approach, and I realized why. This was designed to fail, I think.

    The only way to get the big payoff is if you are in the Top 5% of teachers? It’s not surprising that it would not lead to change.

    Wouldn’t the right approach be to offer all teachers rewards for gain over personal baseline?

  3. Well, it needs to be duplicated perhaps, but I think it indicates something about the canard that teachers are somehow holding back because of pay.

  4. Well, it needs to be duplicated perhaps …

    Well, yes, if we want to take the results seriously. Depending on one small scale study to draw broad based conclusions is a bad idea.

    …but I think it indicates something about the canard that teachers are somehow holding back because of pay.

    It also suggests that raising teacher pay from the current levels won’t gain us (the parents, taxpayers, etc.) any benefit. No?

    -Mark Roulo

  5. Regarding the argument that performance pay would attract higher-quality teachers from other professions, that could also be achieved by simply paying more across the board and being more selective during the hiring process.

  6. But is these were the top 5% of teachers then weren’t they deserving of the higher pay? If they consistently have students with good tests scores why shouldn’t they have higher pay than the teacher next to them that does not? I have never understood why educators are basically all paid the same regardless of their ability. Yuck! Oh yeah….job security and pension are the trade off….must be nice…

  7. Mark: maybe not. If you were to ask me which would improve my teaching, better pay or better training, it would be the better training. In fact, there are many things that would improve my teaching other than pay. Do you find that shocking?

  8. I’ve worked in a school where one year we had merit pay bonuses and the next year we didn’t. It made absolutely no difference to how I taught. I actually worked harder the year I knew I wouldn’t be getting a bonus b/c the kids I had needed more help.

    CEOs and “reformers” only understand greed and can’t understand that not everyone is motivated by it.

  9. MiT-
    You say greed, I say incentive. But, as you correctly claimed, performance pay is not going to motivate good teachers who are already teaching.

    The big failure of this program is that a teacher is not turning out widgets or selling items to customers who come into a store already to buy… instead teachers are required to guide students who mostly don’t want to be there. I don’t want to sound like Malcolm here, but a better comparison to teachers would be prison guards.

    In the end the amount of influence that any one teacher will have on the performance of a student is negligible to the influence that parents, school environment, and curriculum design will have. This focus on teacher performance is nothing more than a an attempt to divert attention to the real problems that plague our schools and society.

  10. You write that Rick Hess wrote his column “before the results were out.” However, it turns out that Hess already knew the results of the study when he wrote his column, although he dishonestly feigned ignorance: http://www.edpolicythoughts.com/2010/09/shame-on-rick-hess.html. Hess wanted his objections to the study design to appear disinterested and unconnected to his own support for merit pay, to the extent he was willing in advance to surrender any rhetorical benefit his side might gain from a positive result. But he was giving up nothing, since he knew the results were negative.

    Nor was this mere omission by Hess. He repeatedly misleads his readers:

    “Whether the merit pay experiment shows big test jumps or none at all, it won’t tell us a damn thing….”

    “The study will confuse the issue, obscure the actual question of interest, and (depending on the results) lend either simple-minded advocates or performance-pay skeptics a cudgel…”

    The fact that Hess failed to make his argument until after he knew the results were negative for merit pay does not prove that his points are wrong. But given his dishonesty here, there is also no reason whatsoever to believe he would have written the same column had the results turned out the other way.

  11. In fact, there are many things that would improve my teaching other than pay. Do you find that shocking?

    No, I don’t. I don’t see much of a constituency pushing for those other things, though 🙁

  12. I have an idea! How about giving the parents of children who score particularly well on these exams a bonus. Perhaps they would become more involved in their children’s education by encouraging them to do homework, switching off the television and video games, and holding their children accountable for part of the learning. When I was a youngster in school, if I did poorly in school it wasn’t the teacher’s fault. It indicated that I wasn’t working as hard as I could. What has happened since then. As a public school teacher with 27 years of experience, I will tell you that I am working as hard as I can. Many of my students are working as hard as they can too, and they are scoring well on the standardized tests we have them do. Many students however, give very little effort to their education despite all of my efforts to encourage them to learn. Many of these students are dealing with incredibly difficult situations at home, and do not have the support they need from their parents. My motivation for teaching is not the substandard pay that I do receive. I do it because I believe in my students, and I want to help humanity. It saddens me to see how politicians use education and dedicated teachers as their “whipping boy”.

  13. Roger Sweeny says:

    In fact, there are many things that would improve my teaching other than pay./i

    Lightly Seasoned, you are so right. Some of my classes consist mostly of ninth graders. They would learn much more if they could all read, write, and do math at a ninth grade level.

    Alas, that doesn’t seem to be the criterion for graduating from middle school.

  14. Roger Sweeny says:

    If we need better teacher, don’t we need better ed schools?

    Homeschooling Granny, that is one of the most profound things written by anyone about education reform–this year or any other.

Trackbacks

  1. […] Merit pay fails a test — Joanne Jacobs […]