We need more tests, but what kind?

American Schools Need More Testing, Not Less, writes Ezekiel J. Emanuel in The New Republic. Students learn more when they take frequent, short tests.

A young neuroscientist named Andrew Butler has gone further, showing that testing can actually facilitate creative problem solving. In Butler’s research, undergraduates were given six prose passages of about 1,000 words each filled with facts and concepts. (Fact: There are approximately 1,000 species of bats. Concept: how bats’ echolocation works.) He had the students just study some of the passages; others, he repeatedly tested them on. Not only did his subjects demonstrate a better grasp of the tested material, but they also fared better when asked to take the concepts about which they’d been quizzed and apply them in completely new contexts—for example, by using what they’d learned about bat and bird wings to answer questions about airplane wings. When students had been tested on the passages, rather than just reading them, they got about 50 percent more of the answers correct. They were better at drawing inferences, thanks to the testing effect.

Only tests written by teachers are useful, responds Diane Ravitch. “Today’s standardized tests are useless.”

What he really admires, and appropriately so, are the regular weekly tests that he took in high school chemistry. His chemistry teacher Mr. Koontz knew what he had taught. He tested the students on what they had learned. He knew by the end of the day or over the weekend which students were keeping up and which ones were falling behind. He could act on that knowledge immediately to make sure that students understood what he thought he had taught and to explain it again to those who did not. He also learned whether to adjust his style of teaching to communicate the concepts and facts of chemistry more clearly to students. Mr. Koontz used the tests appropriately: to help his students.

Standardized exams are being used as “a ranking and rating system, one that gives carrots to teachers if their students do well but beats them with a stick (or fires them and closes their school) if they don’t,” Ravitch writes.

Most researchers say that teacher quality cannot be reliably measured by student test scores, because there are so many other variables that influence the scores, but the federal Department of Education is betting billions of dollars on it.

The job of writing, grading and analyzing tests belongs to “Mr. Koontz, not to Arne Duncan or Pearson or McGraw-Hill,” concludes Ravitch.

About Joanne

Comments

  1. I’d introduce Mr. Emanuel to the concept of feedback but he seems to have stumbled onto the idea himself. I imagine for anyone steeped in the myth and lore of the public education system feedback has to be a foreign, and arcane, concept so kudos to Mr. Emanuel and welcome to the nineteenth century.

    It’s an understandable situation. After all, for feedback to be important the output has to be important and education isn’t important within the public education system. So testing, to the extent it occurs, occurs for purposes other then the refinement and improvement of the public education system and its personnel. More simply, why bother?

  2. Roger Sweeny says:

    Diane Ravitch is half right. If Mr. Koontz gives frequent tests that test what he has taught, and if he then gives term grades that fairly reflect how the students have performed, he will come up with a fair term grade relative to what he has taught.

    But two problems: Different teachers will teach different things so the grades aren’t comparable. And lots of teachers will also grade on classroom behavior, completion of work, “I don’t want to fail too many students,” and a million other things. Often, the two interact: at a school where many students are unprepared or not academically inclined, the course will cover a limited number of topics and have low standards so as not to fail a large number of those students.

    To the extent that grades are needed for “ranking and rating,” e.g., has the student learned the state standards for this course, standardized tests are a necessity.

    • Roger: Your second paragraph is on the money. That’s exactly what happened in the LA Times story linked on this site, where the #2 student in a low-performing HS was obviously vastly unprepared for Berkeley. It also explains the poor scores on last year’s Montgomery County, MD’s HS end-of-course exams. Schools with many underprepared students water down the course and inflate the grades. Only serious testing of material supposed to be covered in the course will reveal what was actually (not) learned, because grades are too easily manipulated. Even in “good” schools it happens. A classmate of my son received an A in their Honors algebra II course, despite having no grade above C and many grades below C on tests and quizzes, because she (more likely parents) had done all her homework. My son received a B+ because he hadn’t done all of his homework, even though he had no grade below 95 on his tests and quizzes.

      • Educationally Incorrect says:

        When I first started teaching HS physics and I sat down to grade my very first homework assignment, I picked the first piece of paper off the stack and started to grade. I found the work perfectly awful and gave, if memory serves, something like 2 or 3 out of 10. After looking at the following homeworks it was clear that the first was, relatively, exemplary. What grade to give the remainder?

        Sure, the non-teachers out there claim that you should give them all the grade they deserve. All Fs. Tried something like that (with a test, not homework). Got called in by the boss, and told, in no uncertain terms, that “if the students don’t know it the teacher didn’t teach it.” or “if the students didn’t feel like doing the assignment then the teacher wasn’t inspiring enough”, or something to that effect.

        In time I simply graded homeworks on a scale of 6-10 so as to avoid certain dismissal.

      • Deirdre Mundy says:

        Math teachers are notorious for this nonsense. The worst were the ones where the bulk of the grade was your end of the term notebook check, where you had to produce every piece of work from the whole semester.

        As a kid who was good at math but lacked organization skills, it meant I could ace all the tests and quizzes and still get a B-/C for the classes. On the other hand, at least Achievement Test (now SAT 2) and AP scores let colleges know what I’d actually learned!

    • palisadesk says:

      “And lots of teachers will also grade on classroom behavior, completion of work, “I don’t want to fail too many students,” and a million other things ”

      Well, maybe where you are. My district does not permit this. Teachers cannot use homework, behavior, participation or the bell curve for grading. Only work done at school counts, and only actual WORK counts (tests, assignments, presentations, etc.) “Participation” doesn’t count towards grades, nor does homework but both undoubtedly affect how much a student actually learns and retains.

      Grading policies are rarely set by individual teachers, more often at the district level.

      The testing issue is complex. Typical standardized (norm-referenced) tests are poorly suited to measure classroom learning, although ones written for specific courses statewide (as in some of the NY Regents from what I understand) can possibly do this. Efforts were made during the 80′s as I recall to develop tests that could measure specific skill and knowledge mastery at the 1-8 level but proved unwieldy and inaccurate.

      Few teachers have training in developing valid and reliable test instruments, so classroom testing, though important, is not going to be consistent. For progress monitoring how students are learning, the growing trend towards curriculum-based measurement (you can google it) is promising.

    • Florida resident says:

      Hello, dear Roger Sweeney.
      Around September 6, 2013 You wrote this about the book by
      Robert Weissberg “Bad Students, _not_ Bad Schools”:
      “It sounds intriguing. My local library network (about 30 libraries) has only one copy, which suggest to me that it is either a bad book or politically incorrect, or both. Using Joanne’s “Search this website …” box indicates she hasn’t posted anything about it. Anyway, I’ll take a look.”
      - See more at: http://www.joannejacobs.com/2013/09/from-superman-to-teach/#sthash.dgB425r2.dpuf
      Did you have a chance to read at least some part of the book ?
      With invariable respect of Ms. Joanne Jacobs,
      your F.r.

  3. cranberry says:

    He writes, “It would be fine for students to have access to the queries in advance, since the sheer size of the database would make it impossible to prep for the resultant tests through brute memorization.”

    That’s not education as I understand it. Memorization of facts, of “the right answer,” is not sufficient. It’s possible to memorize textbooks (or online question databases) without understanding the concepts covered. Read _Surely You’re Joking, Mr. Feynman_, for his experiences of science education in Brazil.

    He’s naive about the effects of such a system. A classroom in which the teacher forces students to memorize the answers to five questions over a week, then tests them, could comply with the system. Unfortunately, a student who understands a process will do better on a test than a student who’s memorized word-for-word the answer to a given prompt.

    Pundits should butt out of education. Choose educated, smart teachers, support them, and stop imposing systems on them without their consent.

    • Your last paragraph seems pretty extreme. Do I correctly understand that you think we should just give teachers all the money they ask for and never check to see whether the expected learning is actually happening? No oversight at all? You must know a lot of stellar teachers. Sadly, they are not all superhuman, and nearly every employee in the world requires some degree of oversight and form of accountability.

      • cranberry says:

        Last I checked, pundits weren’t in charge of school systems. The opinion piece in question was advocating more testing, on top of the state tests administered to most grades every year, in the blind hope that it would “improve learning.”

        It takes time to develop good curriculum. Chasing after every education fad that comes along disrupts the effort to build lesson plans. For example, I think “flipping” the classroom might be a good idea for some classrooms, for some teachers. However, right now, I’m hearing parents complain about teachers’ attempts to “flip” their classrooms. It seems many teachers don’t know precisely what flipping means–or they’ve all found different definitions, from different online sources and professional development providers. Some will be very good, some will be very bad. However, I’m pretty sure if the opinion piece above gathers steam, some teachers will be told by their principals to stop “flipping” and start “testing, testing.” Until the next education fad comes along…

        Oversight is good. It would be good if principals could make more unannounced classroom visits. Requiring teachers to test more frequently than the teachers feel is necessary isn’t guaranteed to improve student learning. It won’t be a form of accountability. Producing ever more data for higher-ups to analyze doesn’t leave much time to create lesson plans or give students feedback on their work.

    • Educationally Incorrect says:

      “Choose educated, smart teachers, support them, and stop imposing systems on them without their consent.”

      Whose definitions do we use?