Forge ahead

High-stakes tests are valid measures of learning, writes Manhattan Institute researcher Jay Greene in Newsday.

Another argument we hear from school officials is that accountability tests don’t promote real learning; they only promote test manipulation. Teachers allegedly “teach to the test” in ways that produce higher scores without conveying knowledge and skills.

To see whether this is the case, I recently performed a nationwide study comparing the results schools got on accountability tests with the same schools’ results on widely respected tests that aren’t used for accountability purposes. These other tests are nationally recognized as genuine measurements of student learning, but they aren’t used for accountability purposes. I found that schools’ results on the two types of tests were highly correlated, indicating that accountability tests do measure real learning and are not distorted by test manipulation.

We also hear from officials at comfortable suburban schools that accountability puts an unnecessary burden on them. Our students are mastering basic skills just fine, they say, complaining that a testing regime would force them to spend less time on science, art, and other things they prefer in order to spend more time on basic skills.

This one sounds fishy from the start. If these schools’ students are really mastering basic skills so well, why would the schools have to change their curricula in order to produce passing grades?

The answer is that NCLB looks at the achievement of low-income, non-English-fluent and minority subgroups; a suburban school’s average score may be high while disadvantaged students are doing poorly.

About Joanne

Comments

  1. You are absolutely right! There’s nothing sillier than listening to educators and parents in places like Scarsdale cry about how test prep will eat up valuable education time. If these kids are so damn smart and well-educated, they should whip through the tests with no problem.

    Also, here in my midwestern corner of privilege, we have the second problem you mentioned. Our schools have looked great on paper for years because of the high average test scores. NCLB required them to disaggregate minorities, and what was revealed was the yawning achievement gap between blacks and whites.

    NCLB now requires the schools to figure out how to help black students achieve. Yeah. It’s about time, I think.

  2. ariztophanes says:

    I like to say, TEACH TO THE TEST: It’s a reading test. TEACH TO THE TEST: It’s a math test. If when the kids are done they can’t pass the test, you may have taught, but you didn’t teach reading or math.

    I would like to know what in these tests is so arcane that decent instruction in math or reading wouldn’t give the kids the tools they need to ace them.

  3. If these kids are so damn smart and well-educated, they should whip through the tests with no problem.

    You’re not looking at the problem correctly. Let’s say you’re in a community were 80% of the students will score the top mark with minimal “teaching to the test”. If the school is to boost its standing, it’ll need 95% percent to score the top mark. So, instead of making teaching a curriculum geared towards the 80%, it’s basically concentrating all its teaching resources towards the remaining 20%.

    For parents of the 80%, school has become a lot less useful for their children.

    Now do you understand? I’m not defending it, but I there’s no doubt that it forces schools to concentrate their resources on the worst performing students, possibly to the detriment of the better students. This is both NCLB’s strength or weakness, depending on where you stand.

  4. Tom, good point. You remind me to write about this soon on my blog.

    But…the “teeth” in NCLB is Title I funds. A school can opt out of receiving them and then not be subject to the punitive side of NCLB. Title I funds are federal dollars given to schools with poor kids to supplement their education.

    So how much Title I money does Scarsdale get? None? A tiny bit? If it’s such a hassle to get kids to improve, they just turn away from the piddling federal money and get on with whatever they do.

    You know, the entire state of Utah has declined to participate in Title I and decided to reject the federal money.

    Finally, in almost every large statistical study I’ve worked with, rich kids do well regardless of the school they go to. Also, high achieving kids. Now, it’s quite likely that rich kids would do better in a school full of other rich kids, rather than a school full of poor kids. But I think that there’s less reason to worry about the achievement of high SES kids. They are not the ones most likely to drop out or fail.

  5. Mike in Texas says:

    Jay Greene writes:

    To see whether this is the case, I recently performed a nationwide study comparing the results schools got on accountability tests with the same schools’ results on widely respected tests that aren’t used for accountability purposes.

    And where is the data?? It isn’t in his article. I wonder if it involves some assessments he invented like his “Education Productivity Index”?

  6. Mike in Texas says:

    It’s interesting that Joanne chose to post this story, which blasts schools for lack of minority achievement, and then the next day posts an article which blames the black achievement gap on cultural issues (acting white).

    Joanne, were you trying to be fair or did you not realize the contradictory nature of these two articles?

  7. Indeed, while you don’t *have* to participate in NCLB, I doubt that decision is left to individual schools. Opposition of this sort to NCLB is usually restricted to the higher performing schools. Secondly, this isn’t necessarily about NCLB, but about accountability and measurement in general.

    In states (and provinces) where schools are rated, they are almost always rated on the proportion of students that reach minimal standards (often 2 years behind the official curriculum). In other words, schools are once again judged by their ability to elevate their least successful students, instead of improving students who are already “maxing out” their test scores.

    Again, I’m not going to defend either side because I can see both side’s arguments very clearly. Do we cancel AP classes to provide more remedial teaching for underachievers? Do we throw away programs that lower drop-out rates to provide help the best student’s reach their potential?

    But I think that there’s less reason to worry about the achievement of high SES kids.

    Depends on how you measure achievement. If you’re simply rating minimal competency on a 1-4 scale (for example), then the better off students will probably score 4 out of 4 no matter what. The question for the parents of such kids is what potential unmeasured knowledge and academic interest is lost to help the lesser achieving kids.

    A better test for those parents would be a 1-1000 test at a university level curriculum. Sure, the average might be 100, but then for a school, it would be more rewarding for the school to raise 50% of the student’s achievement from 200 to 250 than to raise 20% of the student’s scores from 50 to 100.

    Of course, an NCLB-type program that breaks down achievement into specific demographics means that simply raising the average will not be enough, but currently areas that publicly rate schools usually do so using a single average.

  8. Mike in Texas wrote:

    I wonder if it involves some assessments he invented like his “Education Productivity Index”?

    As opposed to the MiT index which consists of:

    Current funding level * some big number = not enough money next year

    Joanne, were you trying to be fair or did you not realize the contradictory nature of these two articles?

    Where’s the contradiction? Are you laboring under the assumption that “acting white” and lousy schools are somehow mutually exclusive? I’d be interested to know what thought processes brought you to that conclusion.

    Tom West wrote:

    In states (and provinces) where schools are rated,

    One of the aspects of the NCLB I haven’t heard anyone comment on is its focus on schools. Not students, not teachers, not districts, but schools. Makes me wonder what the unintended consequences of that focus are liable to be?

    Since this isn’t a topic likely to kick off English-soccer style riots, I’ll take a stab at an outcome.

    One likely outcome is that the authority of the district will be undermined whether the disctrict’s schools manage to comply with NCLB or not.