Evaluating teacher evaluations

Brookings’ new report, Passing Muster: Evaluating Teacher Evaluation Systems, looks at how districts evaluate teacher excellence. The report explores “how a state or the federal government could achieve a uniform standard for dispensing funds to school districts for the recognition of exceptional teachers without imposing a uniform evaluation system on those districts.”

Smart people, confusing study, writes Jay Mathews.

Indeed.

About Joanne

Comments

  1. I have never understood the problem with evaluating teachers. Really. Get a group of grandparents and a teacher from another school. Walk around the building. In a half a day of looking around and talking to students, janitors and other teachers, a sane and competent person can tell you the good teachers from the bad.

  2. Michael E. Lopez says:

    I completely agree with you, mswidget, that the method you describe is an accurate way of determining quality. I can’t count how many times I’ve said that all you have to do to find the good teachers is ask the students, for instance. The knowledge is out there.

    The problem that people are facing isn’t finding a measuring system. It’s finding a measuring system that isn’t going to suffer catastrophic warping from its own deployment in the presence of policy decisions being based on it. Think of it this way:

    Let’s say you have your group of grandparents.

    Situation A: They go to the school, give the principal a list of teachers to retain and dismiss, and the Principal gives them an “A” because they’re absolutely right, but does nothing. That works — the measurement is accurate because all that’s going on is measuring.

    Situation B: They go to the school, where it has been announced that the grandparents will be deciding who gets fired and who gets a raise. The grandparents get a lot of free cookies, guest passes to day spas, and outright bribes. Some teachers refuse to “tattle” to the grandparents out of professional loyalty. Other groups of teachers form “high praise” cartels. The janitor remembers that the last janitor had his car keyed for saying unnice things.

    The measurement fails.

    The object, I take it (and I’m not endorsing this as a goal, merely pointing at it) is to come up with an evaluative scheme that can’t be “gamed” so easily.

    Good luck with that.

  3. Mark Roulo says:

    “The object, I take it (and I’m not endorsing this as a goal, merely pointing at it) is to come up with an evaluative scheme that can’t be ‘gamed’ so easily.”

    I think this is correct.

    Yet industry, in general, manages to evaluate employees in spite of this problem. Far from perfectly, of course. But still, on balance better than the implied evaluation provided by tenure (more years = better). How does industry manage to not have the evaluation gamed such that the evaluation becomes worthless?

  4. Michael E. Lopez says:

    I’m not sure it doesn’t… but at the very least, the person evaluating you often has strong personal incentives to get it right and not allow his or her judgment to be occluded by extraneous factors. My own experience is limited to law firms, really, but if a partner is in charge of reviewing your work and reporting back to the partnership about whether you’re a good associate or not, and he says you are when you in fact are not, at some point there’s a high likelihood that one of the following three things is going to happen:

    1) You’re going to screw something up and cost the firm a client. You’ll be fired, but the partner will look like a dummy for trusting you.

    2) You’re going to have to have your billings reduced because you’re spending too much time on things. This is going to directly hurt the partner’s (and partners’) bottom line and that’s not going to be tolerated for very long.

    3) You’re going to go up for partner at some point and people are going to take a REALLY close look at the work you’ve done and if it’s extremely at odds with the evaluations you’ve gotten, well, questions will be asked.

    Most evaluations involve at least two partners, probably to keep personal preferences from playing too large a role.

    So here’s the question: how is this different than public schools? Well, schools don’t have quality monitors the way firms do. Firms have their clients, who can take their business elsewhere. Clients know what they want, and if they’re not getting it from Dewey, Cheadam & Howe LLP, they can go to Marcus, Down, Anyer & Genda, LLP.

    Sure, we give students tests. That measures something. But what happens if the school fails? Not a whole lot, really. The school gets a bad reputation, maybe the staff gets shuffled around a little… but not much happens at all. The client base is pretty much captive, and the Principal is never going to lose his or her HOUSE because the school is going under because some dipsh*t teacher screwed up and lost 10% of the school’s business.

    It just doesn’t work like that. If it did, the Principal would have a lot more incentive to get teacher evals right.

    I’m just pointing to the differences. I don’t know how to fix it. I have some ideas, surely… but they’re just ideas. Vouchers are nice. But people don’t seem to want them — I mean, really: if people wanted vouchers in any great numbers, there’d be vouchers. It’s a simple matter of elections.