Test answers are in the (missing) book

Pennsylvania’s state exams can be “gamed” by a “shockingly low-tech strategy,” writes Meredith Broussard, a Temple professor of data journalism. All it takes is reading “the textbooks created by the test makers.”

Poor Schools Can’t Win at Standardized Testing because they don’t have the right books, she writes in  The Atlantic.

On  the 2009 Pennsylvania exam, third-grade students were asked to write down an even number with three digits and how they know it’s even.

Here’s an example of a correct answer from a testing supplement put out by the Pennsylvania Department of Education:

This partially correct answer earned one point instead of two:

Everyday Math’s third-grade study guide tells teachers to drill students on the rules for odd and even factors and be able to explain how they know the rule is true, Broussard writes. “A third-grader without a textbook can learn the difference between even and odd numbers, but she will find it hard to guess how the test-maker wants to see that difference explained.”

I’m not shocked that tests are aligned to textbooks. What’s truly disturbing is Broussard’s research into whether Philadelphia schools have the right books. She found district administrators don’t know what curriculum each school is using, what books they have or what they need.

According to district policy, every school is supposed to record its book inventory in a centralized database called the Textbook Storage System. “If you give me that list of books in the Textbook Storage System, I can reverse-engineer it and make you a list of which curriculum each school uses,” I told the curriculum officer.

“Really?” she said. “That would be great. I didn’t know you could do that!”

Principals use their own systems for tracking supplies and books. Short of support staff, schools stack books in closets and forget they’re there. Teachers scavenge materials from closed schools and spend their own money to supplement their $100 a year supplies budget.

Broussard built a program, Stacked Up, which found the average Philadelphia school has 27 percent of the books it needs. But that’s just a guess because nobody really knows who’s got what.

About Joanne


  1. Sigivald says:

    Certainly “cause, it’s an even number” is not an explanation as to … how you know it’s even, or that you actually understand “evenness”.

    I can think of various ways to explain the concept, but stating that the number is even doesn’t do that, and has little to do with however the textbook might have explained it…

  2. The correct answer isn’t even correct. You could divide some odd numbers into three or five “equal parts.” It should have been, you can divide 932 into two equal parts.

  3. Richard Aubrey says:

    Having studied psych half a century ago, we learned that writing an effective question is a combination of hard labor in the hot sun and brain surgery. There are a million ways to get it wrong and the problem is…you know what you want to say so you think you said it.

    As to textbooks, some years ago, Detroit teachers complained, about Thanksgiving, that the books they’d requested were still in the warehouses. The admin wanted to kow who leaked the info.
    Later, the Catholic schools returned a ton of books they thought they’d bought legitimately from the Detroit system, but which were stolen by the warehouse staff and then sold.

  4. Any whole number divisible by 2 and which leaves no remainder is EVEN, otherwise it is ODD.

    This question is pathetic by definition, since that’s something I learned in elementary school back in the early 70’s and having had to write snippets of code to test for EVEN or ODD is quite simple indeed.


  5. The exception to this is of course the number ZERO, since it is not even or odd 🙂

    • Obi-Wandreas says:

      Zero actually is even.

      The one-point kid was lucky. In New York State, a correct answer arrived at by an obviously incorrect method gives zero points, more than a bald answer would. The reasoning, with which I agree, is that if you give no reasoning, we give you the benefit of the doubt. By using lousy reasoning, however, you have proven that you don’t know what you’re talking about. It’s a controversial rule, but to use an extreme analogy, if you walk up to some random person and shoot them, we don’t give you points when they happen to turn out to be a terrorist leader. Stumbling on the correct answer while trying to be wrong does not count as right.

      I have seen a lot of really badly written test questions in my day. This one, however, I think is perfectly fair.

    • Yeah, 0 is even. -You’re probably thinking of +/-. Zero is neither positive nor negative; hence the distinction between positive numbers and nonnegative numbers. Likewise with negative numbers vs. nonpositive numbers ~

  6. Stacy in NJ says:

    Here’s the thing about not having the right book to reference for student prep: wouldn’t a competent principal know to do well on the test teachers and students should have access to appropriate prep material?

    If you’re a principal and you’re NOT supplying the right prep book then you’re a friggin idiot and you deserve to lose your job.

    • Mike in Texas says:

      Why do you think principals get to choose the books their students are using? That’s usually a district decision, based on which textbooks have state level approval.

    • Not necessarily.

      If state/district policy doesn’t give principals a stake in the test outcomes then why should they exert themselves to make sure the learning necessary to post good numbers on the tests occurs? You’re assuming an intrinsic motivation to ensure learning occurs and that that motivation trumps all others.

      But principals do have other motivations then that the kids learn and if they aren’t rewarded for good test scores and/or punished for bad test scores then test scores are irrelevant to the principal.

      You can certainly find principals who do possess the internal drive you assume but that quality’s obviously not widespread.