Best, worst teachers can be spotted early

The best and worst teachers can be identified in their first two years in the classroom by value-added analysis, according to a working paper by University of Virginia and Stanford researchers. Teachers improve as they gain experience, however early effectiveness ratings predict how teachers will be evaluated after five years in the classroom, the study concluded.

Researchers tracked new fourth- and fifth-grade teachers in New York City, analyzing their students’ achievement-test results in math and English/language arts, as well as gender, ethnicity, home language, poverty, special education status, and absences and suspensions. Teachers’ value-added ratings — their ability to improve students’ achievement — in the first two years were compared with the next three years.

Overall, the teachers improved significantly in their first two years in their value-added score. While more than 36 percent of teachers were rated in the lowest of five levels of effectiveness at the start of their careers, only 12 percent were still rated in that same quintile by their third year of teaching.

However, when teachers at each initial level of effectiveness were tracked individually over time, their growth was much less significant. Compared with other teachers who started at the same time they did, teachers in the lowest 20 percent were still likely to be in the lowest 20 percent three to five years later.

Firing teachers in the lowest 10 percent in value-added effectiveness after two years would eliminate 30 percent of the least-effective group in five years, pointed out Tim R. Sass, an economics and public-policy research professor at Georgia State. Principals wouldn’t lose any teachers who’d eventually be rated in the top 10 percent.

About Joanne

Comments

  1. So what really makes one teacher more effective than another if both have similar intellect and content knowledge? Should teachers think of themselves as “on loan” to a school district? No more job security or stability. We’ve moved the goal post for schools and teachers.

    • I can’t say I’m surprised by this. I’ve always viewed successful teaching as a combination of several factors, including content knowledge, ability to clearly communicate and relate with students, and their underlying personality. I have found most teachers do not significantly change these factors after starting their career.

    • GoogleMaster says:

      Why should teaching be different from any other profession? In my profession, employees are usually on a 60-day, 90-day, or 6-month probationary period.

      Plus, I live in a right-to-work state where the employee-employer contract can be terminated for any legal reason.

      • Richard Aubrey says:

        Everybody, including other teachers, know who’s the worst teacher in the school. Firing the worst one every, say, two years would be a problem because…?

        • Nonsense. If everyone “knew” who the worst teacher in the school was then they’d know how to teach themselves and blogs like this wouldn’t even exist because the same “everybody” would be doing a great job and we’d all be happy. What teachers “know” is the difference between what was spewed in ed school — nonstop engagement, hands-on-minds-off activities, and nonstop groupwork — and what they see or don’t see some colleague (or subordinate) doing or not doing.

          • A poll of the teachers in my building would identify a lot of the same teachers as good or bad, and it would have little bearing on if they followed ed school teachings or not. Some of our “best” teachers don’t follow all the ideas from ed school, and some of the “worst” follow them quite a bit. It is not nearly as simplistic as you make it seem.

          • Educationally Incorrect says:

            @Paul

            So tell me more of the theoretic “poll” you speak of. Do we confine ourselves to English teachers judging English teachers and math teachers judging math teachers, etc, — or is it a mixed bag?

            Physical science teachers have all taken English, know how to read, and can relate to the themes of Shakespeare’s plays, while English, art, and social studies teachers are often complete science and math idiots. How is the latter group supposed to judge the former without falling back on ed school crap?

            Why, for example, was I asked in interviews how I would “combine physics with LITERATURE?”

            Former English, art, and social studies teachers are far more common in the admin ranks than anyone else. The people they get rid of are the ones who will get your kid through a freshman engineering class and the keep the entertainers who “combine” hard sciences with “literature”

      • Mike in Texas says:

        Do you want your child’s teacher to be able to stand up and speak up for what’s right for your child?

        Instead of complaining about teachers having due process rights in your state why not spend your time advocating for EVERYONE to have those rights?

        • Richard Aubrey says:

          MiT.
          To be fair, then, our rights would not be trampled by mandatory attendance at school, right?
          Parents whose kids are getting lousy education have priorities, which don’t include, say, killing barred owls so that spotted owls will have a habitat and we can justify having ruined a bunch of logging jobs to save the non-habitat of the former.
          And a teacher who speaks for what’s right for my child is in more jeopardy than one who follows the protocols for whatever is happening. Imagine a teacher strongly supporting the Second Amendment in school. Hell, or the First, without consequence.
          Me either.
          Priorities, Mike. Priorities.

  2. Here’s a hint. If you look around the staff meeting and can’t instantly spot the worst teacher in the school, it could be you!
    Worst teachers are frequently capricious, grading based on how much they like a student and changing the rubrics to ensure their favorites get A’s and the surly but brilliant kid in the back gets a C or below. They have constantly changing expectations, poor vocabularies, and no grasp of the subject matter. The worst teacher in the school often ‘teaches’ wrong information, yet is utterly impervious to correction from students or colleagues. She’s sure that because she completed an Ed program, she is infallible. She loves collaboration and jargon and hates objective assessment. She’s the one who says such outrageous things that students literally run to their next period in order to report the statements to another teacher and trigger a migraine from the stupidity. Students either dread or mock the worst teacher. They frequently complain the the class actively makes them stupider and worse students – and their test scores back them up.

    Worst teachers rigidly adhere to lesson plans developed years before, even when they’re clearly not working. They avoid grading at all costs. They love student presentations, since these can allow them to forgo teaching for months at a time. They frequently screen random movies unrelated to the curriculum. If you saw Twilight in class, it was probably under the direction of a ‘worst teacher.’

    In general, these educators are tenured and un-fireable. Students and staff simply mark the days to their retirement.

    • Educationally Incorrect says:

      >>They avoid grading at all costs. They love student >>presentations, since these can allow them to forgo
      >> teaching for months at a time

      First of all, the above describes my entire M.Ed. program quite accurately and completely. Second, the above almost guaranteed a good evaluation from the assistant principal.

    • Richard Aubrey says:

      Recall talking to a jr. hi. prin about a horrid teacher, showed up hungover, that sort of thing.
      “He’s got to retire some time.”
      Fortunately, the teachers in the system knew who this guy was and made sure their own kids got somebody else. Leaving him to the unconnected.

      • Educationally Incorrect says:

        The problems in education aren’t due solely to a bunch of hung-over child molesters running around with no one being able to fire them. The problems are far more systemic than that and that is why the fire-the-bad teachers-because-we-know-who-they-are will solve nothing.

  3. “The problems are far more systemic than that and that is why the fire-the-bad teachers-because-we-know-who-they-are will solve nothing.”

    It will prevent some students from getting stuck with a teacher who believe Puerto Rico is a country, that Marx was a Soviet leader, that squaring a number is the same as multiplying it by two, or who just doesn’t want to come to work on Mondays or Fridays.

    • Educationally Incorrect says:

      >> It will prevent some students from
      >> getting stuck with a teacher who believe
      >> Puerto Rico is a country, that Marx was
      >> a Soviet leader, that squaring a number
      >> is the same as multiplying it by two, or
      >> who just doesn’t want to come to work on
      >> Mondays or Fridays

      And just what is it that permits such people in classrooms IN THE FIRST PLACE?

      And, if such people are running around in classrooms, what makes you think that their colleagues and superiors, coming from the same ranks, will actually fire said teachers for the very reasons you cite?

      By your reasoning we should fire airline pilots who don’t know what a stall is instead of addressing such concerns SYSTEMICALLY, in training, LONG before such ignorance is licensed to fly passengers.

      • Richard Aubrey says:

        Yeah, well we should. Should have. Didn’t. Now what?
        Hey! I have an idea. Do both.
        But, if you’re looking for the quickest effect, which would be the choice?
        I had some damn’ fine teachers in HS, except for one, who was not all that interested in the material. We had a class called “common learning”, which was two hours a day, grades 7-11, combining history, lit, and composition. Having a loser there cost big time. Not to mention the teachers for the next year.

      • Educationally Incorrect says:

        @Richard

        >> Yeah, well we should.

        Really? So if you’re the guy who let incompetent pilots slip by in the first place we should trust YOU to fix things by going on the firing spree? I don’t think so.

        Usually people on your side of the issue insist that the principal do the identifying and firing. News flash: who do you think hired these people in the first place?

        >> Should have. Didn’t. Now what?
        Translation: we were idiots, so we need to continue.

        >> Hey! I have an idea. Do both.
        Ever think of running for office?

        How do you go about identifying the teachers you wish to fire? I’ve seen a lot of anectotal evidence given in this thread, but how do you translate this into a meaningful strategy. There are 3M+ teachers in this country. Are the teachers who teach evolution the “bad” teachers, or are the ones who teach “intelligent design?”

        And once you fire them, to quote you, “Now what?” Do you think you’ll magically get rid of bad teachers and replace them with good ones?

        >> But, if you’re looking for the quickest
        >> effect, which would be the choice?
        First of all, we don’t know if the effect would be what you think it would be, so your point of “quickest” could very well be moot. Second, blindly seeking quickest is rarely the optimal strategy.

        >> I had some damn’ fine teachers in HS,
        >> except for one, who was not all that
        >> interested in the material. We had a
        >> class called “common learning”, which
        >> was two hours a day, grades 7-11,
        >> combining history, lit, and composition.
        >> Having a loser there cost big time. Not
        >> to mention the teachers for the next
        >> year.
        I’ve had my share of crappy teachers too, but none of that would be remedied by mindless firing.

        • Richard Aubrey says:

          We can always have a third party do the firing: Like a new school board, with a legislated change in tenure. So it wouldn’t be the ed profs, or the principals doing it. In the meantime, the ed schools can be cleaned up.
          But, neither can/will be done so the march to home/charter/private/parochial continues.
          It’s not “mindless” to fire a demonstrably crappy teacher. It’s mindless to fire by, oh, say, a lottery.
          Replacements? Well, among other things, the burnout rate among the teachers who’ve had to deal with the gone teachers’ results will be reduced.
          But, as I say, perhaps there won’t be that much demand.
          See gun-shaped pastries, for example.

    • My brother once had a teacher who graded ‘on the medium.’ When my mom asked for clarification, she explained that grading on the ‘medium’ meant lining up all the numbers and taking the one in the middle–the medium one. (median.) So, my brother, smart aleck kid that he was, decided to get 100 on 51 assignments and 0 on 49. Heck, it meant he only had to work for half the Semester! Unfortunately, he alternated work and not-work. If she’d taken a true median, he’d have come out ahead, but this teacher (she taught science, BTW, so should have had basic math skills) lined up the grades in DATE order and the middle date happened to be one with a zero on it! So, my brother failed the class, all because he wrongly assumed that a teacher who didn’t realize that most teachers use the mean, not the median, would at least be smart enough to calculate the median correctly.

      And then there was my fifth grade teacher, who swore the Northwest Coastal Indian tribes lived in teepees and hunted buffalo, because DUH, that’s what ALL Indiana tribes did! After all, that’s what makes an Indian and Indian! Otherwise they’d be a pilgrim or an Eskimo or something……

      Again, worst teacher. Everyone knew it. They were waiting for her to fail the Praxis I for the 5th time so they could fire her.

  4. l hodge says:

    Look on pg 28 of the report to see how the ratings for 100 randomly chosen teachers change from year to year: http://twitpic.com/c8wzi4. The graph looks to be nearly random.

    It is sort of like saying shorter people don’t weigh as much as taller people. On average that is true. But, there are many, many exceptions. So many exceptions, that the charts of individual teachers on page 28 & 30 almost look random. The charts like those on page 29 are averaging many teachers together, so the results look much more stable.

    • SuperSub says:

      While it does look “random” with the trajectories from all the teachers thrown together into one graph, the point the authors are trying to make is that the trajectories for the teachers on the bottom stay on the bottom. It’s hard to see that with the overlapping trajectories, but the successive graphs demonstrate the point.

      I don’t see why so many are discussing this in the terms of unfair terminations and witchhunts… if anything, this emphasizes the need for effective objective observation of teachers pre-tenure. The new value-added systems do not need to be universal… they can be used to determine tenure. Administrators can focus their attentions on the probationary teachers, producing better quality observations and mentoring instead of being spread across the whole faculty.

  5. Here is a systematic fix. Let parents veto one teacher per year so that their child will not be in that teacher’s class. Suppose a school has four fifth grade classes. If 3/4 of the parent veto one of the teachers, that is a clear indication that the teacher should not be there. At bare minimum, the parents would be happier knowing that their kid is not in the worst option.

    • Richard Aubrey says:

      Or we could look at the one whose classes do not contain other teachers’ kids. Don’t even have to vote.

  6. Richard Aubrey says:

    Herewith an annoyance with a particular teacher:
    Working with a Nepali immigrant on 20th cent US history.
    WWII.
    Fifty-question work sheet. One battle named–Guadalcanal. Not Stalingrad, Normandy, Midway, etc.
    No campaigns. Did have to do some context so he could get his mom’s approval to see Schindler’s List.
    What was the 442d RCT?
    Effects on the economy, women’s place.
    Seven questions on the internment of Japanese Americans, and four or five on race relations.
    IOW, in the entire worksheet about WW II and the US, about 25% was how bad the US was.
    Did a bit of sabotage, got footage of civilians greeting US troops. Told how two towns in Holland have streets named after my Father’s division–Timberwolfstraat, and one named after his division commander, Generaalallenweg. Rose window in church in Ste. Mere Eglise with paratroopers flanking the Holy Mother and the Infant, a street in Carentan named after the 101st Airborne Division. I do what I can.
    But, were I a parent of this kid, I’d have a word with the teacher.
    Nothing about 35,000 dead guys in the Eighth Air Force, but we did talk about the Tuskegee Airmen.
    Got some footage from Private Ryan, going up the beach.
    At least, now, he knows something.
    But what does a regular kid do who doesn’t have an educated tutor?

    • Mark Roulo says:

      If you have not shown “The World at War” (BBC), try to get it (borrow from library if possible).

      A longer list would include:
      *) The movie “Midway” (1975, I think)
      *) Tora, Tora, Tora
      *) A Bridge Too Far
      *) Stalingrad (1993) maybe (I have not seen it)

      • Richard Aubrey says:

        Mark.
        Thanks. I’m a history buff, milhist in particular. I’d love to see some station run Victory at Sea. Did you know you could get the soundtrack, or whatever they call the music less the voiceover? Had that one on LP.
        Problem for this guy is that he has no context and little vocab. Explaining immigrants and political machines in the nineteenth century. “The ward operative[works for the organization] greets the immigrant on the dock with a bucket of coal, courtesy of that nice Mr. Tammany.”
        “What’s coal?”
        Off to Wiki.
        Oh, yeah. Only one bomb fell on Pearl Harbor and Ben Affleck was in it. Sorry.
        Tuskegee Airman. Never lost a bomber.
        How do you lose a bomber? Um. Fighters shoot at bombers so other fighters escort the bombers to fight the other fighters.
        Had an Coastie uncle in the North Atlantic. Said that “HMS Ulysses” is dead on. So I quoted McLean to my kid–anybody who goes past Bear Island twice is never the same.
        I can’t explain Midway without going into Torpedo Eight, which means torpedos and carriers and dive bombers and coordinated attacks and the urge to say, this isn’t working, where are the fighters, let’s go back and get this right, and going in anyway.
        OTOH, I’m not sure he’s all that far behind his classmates.
        My wife had to bring his sister out to our house to get a picture taken and printed out so she could make a baseball card, which is a picture of herself with some info. Goals. Didn’t bother to explain baseball cards, nor try to explain why it’s an assignment in high school.
        I am so glad my wife and I were available to educate our kids after they got home from school.
        There was an old movie about the Bulge with Jack Palance and Buddy Ebsen. Pretty good. James Whitmore’s performance was good in his flick about the Bulge, but a couple too many stock characters.
        Got a relation who went to see where her uncle went in on June 10, in a B26 near Carentan. Seems the locals have a monument, and one nearby where a glider augured in. Each year, they read the names and the kids say–Mort pour la France–or whatever it is in French.
        I guess the locals around Meyerode keep one up about Eric Wood.
        And you can get a tour of the cemeteries from the American Battle Monuments Commission. Families adopt the graves.
        My relation said she saw the locals using scissors to trim the grass around the headstones at the Normandy Cemetery because weedwhackers might damage the stone.
        I suppose if you want to learn about the US in WW II, you could ask the foreigners.
        Just to get sloppy, I found a Make a Wish incident where some Belgian kid, sick as hell, had heard his grandfather talking about being liberated and wanted to be an American soldier. So the foundation shipped him to the US to work with some of our guys.
        “Make a wish” brisbois will probably get it.
        We got my Dad’s Silver Star cite and posted it in his room at the assisted living place. The staff reads it from time to time because, they say, it reminds them that their charges used to be go-getters who could take care of business.
        But my Nepali kid’s history class will probably remember more about the Detroit riots of 43 than, say, Torpedo Eight, or Tarawa or …screw it. I’m getting mad again.

  7. l hodge says:

    This chart from pg 30 gives a visual image of just how inaccurate these models can be: http://twitpic.com/c9noot.

    The points look almost random – doesn’t look like they follow the dashed line very much at all.

    On page 18 of the report the authors reference the above graph when stating that, “the degree of error for individual predictions is substantively large”.

    On average the low scoring remain low scoring. But, as the graph shows, there are many, many exceptions. It is much easier to make predictions about large groups than about individuals.