‘Readability’ is unreliable

PictureStuck in the Middle is a collection of comics that’s way too graphic about middle-school sex and swearing, writes Momma Bear. She advised an irate mother to complain to the school principal.

Then Momma Bear wondered why Accelerated Reader labels the book a 3.0, readable by third graders, and recommends it for grades 4 to 8. Renaissance Learning, which owns AR, explained that content doesn’t count. The company’s readability formula measures text complexity. “Graphic novels tend to produce a lower readability level because of the short sentence structure.”

AR also provides an “interest level” for each book and warns of “objectionable content” in book summaries. “Because of your concern,” the company raised the Interest Level from MG to MG+ (Grade 6 and above), the email said.

Reading formulas are unreliable, writes Dan Willingham on RealClearEducation, his new blog home.

Educators are often uneasy with readability formulas; the text characteristics are things like “words per sentence,” and “word frequency” (i.e., how many rare words are in the text). These seem far removed from the comprehension processes that would actually make a text more appropriate for third grade rather than fourth.

To put it another way, there’s more to reading than simple properties of words and sentences. There’s building meaning across sentences, and connecting meaning of whole paragraphs into arguments, and into themes. Readability formulas represent a gamble. The gamble is that the word- and sentence-level metrics will be highly correlated with the other, more important characteristics.

Only the Dale-Chall formula is “consistently above chance” in a new study,  writes Willingham. It’s easier to assess readability for high-ability than for low-ability students.

About Joanne

Comments

  1. Mike in Texas says:

    Dan Willingham is not an expert in reading, he’s a psychologist. Plus his research relies on the DIBELS test, which relies on nonsense word reading and has widely been criticized.

    http://unitedoptout.com/high-stakes-testing-thy-name-is-dibels/

  2. “UNITED OPT OUT NATIONAL”
    “THE MOVEMENT TO END CORPORATE EDUCATION REFORM”

    Yeah, I can see the stuff posted to that web site being the epitome of scholarly evenhandedness. Maybe the “Occupy” movement has something to say on the subject as well.

    And I’m pretty sure being a reading expert and psychologist aren’t inherently mutually exclusive. Is there some reason you can’t be a psychologist and a reading expert?