‘Readability’ is unreliable

PictureStuck in the Middle is a collection of comics that’s way too graphic about middle-school sex and swearing, writes Momma Bear. She advised an irate mother to complain to the school principal.

Then Momma Bear wondered why Accelerated Reader labels the book a 3.0, readable by third graders, and recommends it for grades 4 to 8. Renaissance Learning, which owns AR, explained that content doesn’t count. The company’s readability formula measures text complexity. “Graphic novels tend to produce a lower readability level because of the short sentence structure.”

AR also provides an “interest level” for each book and warns of “objectionable content” in book summaries. “Because of your concern,” the company raised the Interest Level from MG to MG+ (Grade 6 and above), the email said.

Reading formulas are unreliable, writes Dan Willingham on RealClearEducation, his new blog home.

Educators are often uneasy with readability formulas; the text characteristics are things like “words per sentence,” and “word frequency” (i.e., how many rare words are in the text). These seem far removed from the comprehension processes that would actually make a text more appropriate for third grade rather than fourth.

To put it another way, there’s more to reading than simple properties of words and sentences. There’s building meaning across sentences, and connecting meaning of whole paragraphs into arguments, and into themes. Readability formulas represent a gamble. The gamble is that the word- and sentence-level metrics will be highly correlated with the other, more important characteristics.

Only the Dale-Chall formula is “consistently above chance” in a new study,  writes Willingham. It’s easier to assess readability for high-ability than for low-ability students.

Fifth-grade reading in high school

High school students need to read challenging books to prepare for college and informed citizenship, writes Sandra Stotsky of the University of Arkansas in response to to Renaissance Learning’s 2012 report. High school students’ top 40 books average a 5.3 (just about fifth grade) reading level, according to the company, which makes Accelerated Reader. That’s down from 6.1 in the 2008 report, notes Stotsky.

This republic cannot flourish in the 21st century, no matter how much time English or reading teachers spend teaching “21st century skills” . . . if the bulk of our population is reading at or below the fifth-grade level.

Accelerated Reader software quizzes students on their reading and awards points based on difficulty. The report doesn’t count books without a quiz. But AR now includes a very large number of books.

It’s not just that students choose easy books, Stotsky writes. According to the report, librarians are recommending books of interest to high school students that are written at the fourth- to fifth-grade level.

Readability formulas don’t tell us about the literary aspects of a literary text, but they do provide objective measures of vocabulary difficulty and sentence complexity. And why no serious historical nonfiction?

The list of most frequently read graphic novels shows many high school students are reading “classics” reweritten at a second-, third- or fourth-grade level, Stotsky writes. Examples are: Harriet Tubman and the Underground
Railroad, A Tale of Two Cities, Romeo and Juliet, The Time Machine, A Midsummer Night’s DreamJane EyreDr. Jekyll and Mr. Hyde, The Scarlet Letter and A Christmas Carol.

Only Romeo and Juliet is on the top 40 list for all high school students. “In a few years, struggling readers may be more familiar with the “classics” as rewritten than regular readers are with them as written,” Stotsky fears.

Common Core Standards writer David Coleman also sees a problem: “If you examine the top 40 lists of what students are reading today in sixth–twelfth grade, you will find much of it is not complex enough to prepare them for the rigors of college and career.” The reports includes Common Core Standards’ “exemplars” of nonfiction and fiction books recommended at different grade levels.

A day in the school library

As part of a Stanford alumni day of service, I spent Saturday at an elementary school library helping to mark books with the Accelerated Reader grade level, quiz number and points awarded for doing well on the quiz. The school has quite a few books, most of which have an associated quiz in the AR computer. Kids enjoy earning points so much they sometimes choose high-point books they’re not all that interested in, the librarian told us.

My partner and I coded lots of sports books, lots of Beverly Cleary and Roald Dahl and some Artemis Fowl. Sometimes the reading level or the points seemed odd:  Barbara Cohen’s Passover story, Carp in the Bathtub, was only one point, while similar books were three or four points. (Thirteen was the highest we handled in the C and D authors.)

Both of us questioned whether Sandra Cisneros’ House on Mango Street, which includes a sexual assault on a young girl, belongs in an elementary library. (AR marks it as upper-grade reading with a surprisingly low reading level.) The librarian said she’d take a closer look at the book, which had been donated.

It was fun. I love children’s books.