Do our tests pass the test?

Hiring, retaining and rewarding good teachers is essential to improving education, writes Eduflack. But are the tests we have good enough to use to evaluate teacher effectiveness?

In an era where most of our student assessments are focused on measuring reading and math proficiency in grades three through eight, do we really have a full quantitative picture to separate the good teachers from the bad?  

For one thing, many states don’t measure students’ mastery of science, history, art, music and foreign languages. There’s no data to show how students of specialist teachers perform.
What about the notion of the teacher team?  If I am a middle school student, my performance on the state reading exam is impacted by more than just what is happening in my ELA class.  Hopefully, my social studies teacher is introducing new vocabulary words and forcing me to apply critical thinking and comprehension skills to what I am reading.  My first or second year of a foreign language is getting me to reflect more closely on sentence structure and the roots and meanings of key words or word parts.  Even my math and science classes are contributing to my overall literacy skills.  So if I gain on the state reading exam, is that just a win for my reading teacher (as the current proposals would call for) or is that a win for the entire faculty?  
. . . Are all gains equal?  If I am a math teacher in an upper-class suburban public school, and my students post five point gains on the state assessment, taking them from 92 percent to 97 percent, is that equal to a math teacher in a failing urban middle school who boosts student math performance from 45 percent to 50 percent?  

 Eduflack worries about all the intangibles that help teachers work effectively — or undercut their efforts. A teacher who fails to raise test scores may be held back by an incompetent principal or ineffective colleagues. (I used to think my junior high school looked much better than it should have because we had great parents offsetting some third-rate teachers and a fourth-rate principal.)

As states develop data systems that track students over time, it will be easier to see which teachers (and schools) make a difference, Eduflack writes. It would help to agree on a common set of standards. Using data to evaluate teachers — firing some and paying more to others — is “step 106.”
About Joanne


  1. Homeschooling Granny says:

    Couldn’t psychologists create an interview schedule for students to learn from them who are effective teachers, taking care to distinguish between popular and effective?

  2. I suspect that implementing universal school choice is just as easy technically and politically as trying to solve the teacher quality problem within the current system.

  3. Roger Sweeny says:

    Eduflack is certainly right about the difficulty of determining “merit” in a teacher and implementing a “merit pay” system.

    Surprisingly, he fails to mention that almost every school system in America already has a merit pay system–a system that pays some teachers considerably more than others. This system is based on taking education courses. The more you take, the higher your pay (and pension).

    This even though most teachers will tell you that most education courses are a waste of time.

    Unless eduflack is willing to apply the same standards to the merit system that we already have that he applies to the merit systems that we don’t have, I can’t take him too seriously.

  4. The legislature in the state of Washington, as part of ed reform, is mandating reform of teacher pay (we’re presently on a statewide salary schedule where all teachers are paid the same based on years of service and credits earned…a handful of local districts supplement, but in essence every seventh year teacher with a MA, regardless of where they teach in the state,gets paid the same). The legislature has assigned a task force with the Professional Educator Standards Board to create a top tier voluntary designation called “Master Teacher,” which would bring a pay bump and be highly demanding, akin to NBPTS certification or harder. The idea is that these “master teachers” will demonstrate a higher degree of effectiveness and there may be incentives for master teachers to be redistributed to higher needs schools. Effectiveness, by definitions currently being considered, includes school and community collaboration and leadership, in addition to demonstrable gains in student performance in relation to state and/or national content standards.

    A panel of eight practicing teachers, two administrators, two former teachers and members of PESB has been charged with creating the standards and measures of this “master” teacher to recommend to the legislature. It looks promising, the idea being that the process will be professional development which results in real professional growth and which successfully identifies the most effective teachers amongst the ranks. One of the first conclusions drawn in the research dome by this group: standardized test data is not the best indicator of teacher effectiveness, partly because that is not what those tests were designed to assess. Further, snapshot test scores do not represent student growth over time. Further, there is not statewide test to assess the Japanese or Ceramics students, for example, so the model currently in design would enable specific content area teachers such as these to demonstrate their mastery of teaching.

    I think it is a distinct step in the right direction.

  5. On merit pay…
    Tolja so (comment 11).


    Some years ago I read of an experiment in which a couple of psychologists hired an actor to deliver a lecture on Educational Psychology to an audience of Professors of Education. The psychologists wrote for their actor a lively, dramatic speach with humorous anecdotes and jokes and abundant technical terms to establish the actor’s authority. The larger joke was, the speech contained ZERO information. If somewhere in the speech the actor asserted A, somewhere else he asserted not-A. After the speech, the audience evaluated the “Herr Doktor Von Bullsheisen” (I made that up. Cannot recall what name they gave him) and gave him high marks. Educational experts cannot recognize expertise.

  6. I bet, though, if they watched an actor try to teach a class of kids, examined that actor’s lesson plans and long term unit and year-long curriculum map, probed that actor with questions about intentionality in practice and hypothetical situations for how to craft learning goals and strategies for specific kids, I bet they’d be able to spot an impostor.

    One of the biggest problems with defining the top tier of teacher effectiveness is that it is one of those things can seldom be put into words, but you know it when you see it in action.

  7. Tests scores, and more specifically test cutoffs scores, can be modified to suit the political times.

    When Bush was making his first run for president, the Math score needed to pass was around 58%. With Rick Perry as governor the passing scores for 5th grade Science has been anywhere from 78% to 65%, with the lower number coming in election years.

    NYCEducator has written repeatedly about false gains in test scores in NYC.

    The test is only part of the problem, the rest of the problem is what politicians are doing with the data.