Good teachers, low value-added scores

At a very high-achieving Brooklyn elementary school, the fifth-grade teachers posted low value-added scores, writes Michael Winerip in the New York Times. They’re a talented, hard-working group, says the principal. So what happened?

Though 89 percent of P.S. 146 fifth graders were rated proficient in math in 2009, the year before, as fourth graders, 97 percent were rated as proficient. This resulted in the worst thing that can happen to a teacher in America today: negative value was added.

The difference between 89 percent and 97 percent proficiency at P.S. 146 is the result of three children scoring a 2 out of 4 instead of a 3 out of 4.

. . . In New York City, fourth-grade test results can determine where a child will go to middle school. Fifth-grade scores have never mattered much, so teachers have been free to focus on project-based learning.

If Winerip’s theory is correct, all of New York City’s fifth-grade teachers should have low value-added scores. Or perhaps there’d be an effect only in schools with students who care about getting into a good middle school.

Update: Winerip provides an example of creative teaching:

Using the new curriculum, children work in groups to solve real-life problems. On Friday, each group spent an hour developing a system to calculate who ate more — eight students sharing seven submarine sandwiches; five students sharing four; or four sharing three. Each child developed his own solution, and the group decided which way was best.

. . . This week, students will advance from dividing sandwiches to comparing fractions with different denominators, to calculating least common denominators.

In another fifth-grade class, students have spent weeks writing research papers on the Mayans. Students might score higher, Winerip suggests, if they drilled on writing essays for tests: Write a topic sentence, three sentences that support the thesis with examples from literature, current events and personal experience and a concluding sentence.

I spent my entire high school career writing topic sentences supported by subtopic sentences supported by three “concrete and specific” details. And I wrote a report on the Mayans in sixth grade. Writing research papers and learning to support a thesis with examples are not incompatible.

If Winerip is correct about the numbers — if it’s possible for 89 percent of students to score proficient and the teachers to look like losers — then the value-added system is not reliable.

About Joanne

Comments

  1. smintheus says:

    Characteristically perverse misinterpretation of this damning article, Joanne. It demonstrates why teachers have been teaching to the tests, rather than helping students to develop critical skills. I notice that you also skip over the fact that these test scores now make up some 40% of the teachers’ evaluations. As always testing uber alles, eh Joanne?

    • Roger Sweeny says:

      Most teachers teach to their own tests. How is this any better than teaching to a state test?

    • Because when teachers teach to their own test all teachers are above average.

  2. Three children getting a score of 2 instead of three doesn’t mean much, statisically; but it does make one wonder, if fifth grade has been devoted to “project-based learning” whether that’s really a good idea.

    (Probably not; and essentially never in math, because there’s no way to organize projects so that mathematical concepts and skills arise in the order they need to be acquired.)

    When they weren’t “teaching to the test” they were wasting a whole year for a sizable fraction of their classes.

  3. Situations like this are just the tip of the iceberg. You will have more examples of these systemic failures. And it’s not just 4th grade let-down; you can have move-ins, reorganization, a new teacher last year, many things, let’s call them variables, will affect the testing scores.

    But when you have so many uncontrolled variables in any algorithm, there’s another word for it – chaos.

    And I don’t think we’ve factored in chaos theory to the value added component. But who knows, since the companies pushing this method won’t release their computational methods because it’s proprietary knowledge.

    Just what I wanted; my future decided by a black box somewhere.

  4. If Winerip is correct about the numbers — if it’s possible for 89 percent of students to score proficient and the teachers to look like losers — then the value-added system is not reliable.

    Oh, he’s right. If all your kids come in at 97%, then the only way to get a strong value add is to move them to 95%. Otherwise, you didn’t add value. I’m surprised you didn’t know this.

    This is a problem particularly for high-income elementary schools, because the elementary school tests are quite easy, and your average suburban kid is at 95% whether he goes to school or watches TV all day. Don’t you remember all the teachers in the LA Times posting the same thing, that their kids were very high scoring coming in, so they couldn’t improve?

    And what else could it mean, if not that? That’s what “value add” means, quite literally.

    The problem is that people who are thinking about judging teachers by test scores are thinking of one group, and one group only: kids who are in elementary school and can be assessed by a “grade level” number–below, at, or above–and the kids in question are all at or below. That’s the only type of kids they are thinking of. They aren’t thinking about high achieving elementary schools because no one cares about them. And they aren’t thinking about high school kids, where it’s impossible to test “value add” unless your kids are taking the same course two years in a row, because value-adders are convinced (wrongly) that if we “fix” elementary schoolers who are below average, then all high school problems will vanish.

    So that’s it–the only thing they are thinking of. And their failure to anticipate the problems with this will sink value add.

    • whoops:

      If all your kids come in at 97%, then the only way to get a strong value add is to move them to 95%.

      That should be “come in at 87%”.

  5. >Each child developed his own solution, and the group decided which way was best.

    Um, that doesn’t sound like teaching, that sounds like the opposite of teaching.

  6. Tom Linehan says:

    Joanne wrote: “I spent my entire high school career writing topic sentences supported by subtopic sentences supported by three “concrete and specific” details. And I wrote a report on the Mayans in sixth grade. Writing research papers and learning to support a thesis with examples are not incompatible.”

    Good for you Joanne, You hit the nail right on the head. I do not buy the contention that teaching content and teaching skills are mutually exclusive.

  7. Unfortunately teachers are doing a really bad job such as is the case at PS39 where they score kids based on exams. They actually do not teach at all but demand that parents do the teaching for them. What is the point of going to school then if I have to teach ‘most’ of the material to my child? The last 3 years have been a might mare with incapable teachers who have the nerve to actually complain that ‘hey’ my child is not getting it (but they are not teaching really), where in fact they only concentrate on exams. I have a problem as I try to teach my child the material that teachers do not teach and this takes away the time that children needs to spend on ‘memorizing’ how to pass the test but not actually understand the material. Without understanding, these kids will have a tough time in the future as it is well known that without understanding, the material that is memorized gets lost/forgotten quickly.

    Then we have state wide exams that get scored in the summer (get this, in the summer), so your child can end up in summer school even though he/she passed the actual exam! They end up in the summer school based on teacher recommendation! Horrible system.

    Poor NYC kids and parents too.