Reading and knowing

Reading tests that measure grade-level knowledge as well as skills would help students build comprehension, writes E.D. Hirsch, Core Knowledge founder and author of The Knowledge Deficit, in a New York Times op-ed.

These much maligned, fill-in-the-bubble reading tests are technically among the most reliable and valid tests available. The problem is that the reading passages used in these tests are random. They are not aligned with explicit grade-by-grade content standards. Children are asked to read and then answer multiple-choice questions about such topics as taking a hike in the Appalachians even though they’ve never left the sidewalks of New York, nor studied the Appalachians in school.

Hirsch advocates using reading passages on tests taken  “from each grade’s specific curricular content in literature, science, history, geography and the arts.”

Test preparation would focus on the content of the tests, rather than continue the fruitless attempt to teach test taking.

In a 1988 study, researchers gave strong and weak readers in seventh and eighth grade a reading test with passages about baseball.  “Low-level readers with high baseball knowledge significantly outperformed strong readers with little background knowledge,” he writes.

This reform would push states to set specific learning goals, Hirsch adds. Teaching to the test would mean teaching the curriculum.  Disadvantaged students, who rely on their teachers to teach them knowledge and vocabulary they can’t learn at home, will have a chance to catch up.

About Joanne

Comments

  1. The “literacy question” is key to any education reform, and the problem is far too many people understand the skill and its importance. One who does is a Denver-area researcher and teacher named Cris Tovani. Her book “I Read It, but I Don’t Get It” is integral to this discussion. For far too long, schools have taught decoding and then assigned reading – and that is at the heart of our problem (that and the arbitrary nature of the way we do standardized testing of reading as Hirsch points out). Tovani’s second book “Do I Really Have to Teach Reading” addresses the complexity of this question, is it focuses on reading in the content areas. Too many teachers think literacy is an English skill, not a learning skill. Thus, they neglect to teach their kids how to read science or history or math. Until these issues are addressed systemically, there will be no progress in reading scores and no real progress in education reform. Another book that is relevant is Ellin Keene’s “Moasics of Thought.”

    Tovani and Keene both work with an organization known as the PEBC (Public Education Business Coalition) that is on the cutting edge of literacy, and their work needs much more press. Perhaps the next time Oprah does a book club selection, she could choose Tovani’s and Keene’s books, and the audience could have a discussion about whether they really understand the novels Oprah asks them to read. Research shows many of them don’t.

  2. Andy Freeman says:

    > Children are asked to read and then answer multiple-choice questions about such topics as taking a hike in the Appalachians even though they’ve never left the sidewalks of New York, nor studied the Appalachians in school.

    They can read and answer questions about Harry Potter even though none of them have ever cast a spell. They can read and answer questions about science fiction.

  3. Andy Freeman says:

    My point is that if they can only read and answer based on what they know, are they actually answering from their reading or from their experience? It sounds like the test is revealing that they can’t answer from their reading.

    A test that folks fail is not necessarily a bad test.

  4. Andy,

    The basic idea is that readers use “background knowledge” to access new information. Harry Potter is a novel filled with archetypal images that the kids recognize from books, movies, and TV shows with which they are familiar. That’s how people comprehend much fiction – they make text-to-text connections (or movies). Harry is much like Luke Skywalker and Frodo and even Dorothy from the Wizard of Oz. He is like many cartoon characters and heroes of children’s stories. That, and basic vocabulary issues, is why many kids are at a continual disadvantage in reading assessment.

    That is not to say that the kids can’t learn to accommodate their weaknesses. That’s what effective instruction does. However, as I’ve noted, many teachers are not adept at teaching the skill.

  5. Margo/Mom says:

    Of course, we could play with Hirsch’s assumption by tossing in a number of reading questions based on life in the ‘hood and see if the distribution of scores changes. He may have a germ of an idea, but it is also couched in his deep commitment to providing every kid with exposure to the same set of culturally determined “knowledge” experiences. In the end, I think that this gets into some dangerous territory, and means that someone has to be in charge of sorting out whether Dickens should be read in eighth grade or tenth, and Tale of Two Cities or David Copperfield; and what about Emily Dickenson vs e e cummings or Pablo Neruda? Not to worry, Hirsch has already laid it all out for us in his Core Knowledge Curriculum.

    I suspect that if the reading scores are being heavily skewed by the students’ prior knowledge of the content of the reading selections then nobody is learning to read and understand very well.

  6. This recalls the reading comprehension skills tests I took in school. Often, the subject of the paragraph was a mystery to me, but the answer(s) to the questions were all self-contained in the paragraph, so no outside knowledge was necessary.

    I never had any problems with these. Where lack of context hurts me is in straight knowledge memorization, which is why I am abysmal at foreign languages–there is no context to the words. But the context, or enough of it, for the reading comprehension questions is contained within the paragraphs themselves, so that shouldn’t be a problem.

  7. Rex: Your results are not typical. Experiments going back to the 1960s consistently show HUGE effects of background knowledge on reading comprehension and memory–the more you know about a subject the easier it is to understand new information about it and the better you will remember it later
    Margo/Mom: Hirsch has made his bid for what he sees as important background knowledge that students should know, but bear in mind that if you think that background knowledge is important to reading and listening comprehension, then implicit assumptions are made about this everyday; the editors of New York Times, Newsweek, television shows, etc. must decide which references need to be explained and which one they can safely assume that their readers/viewers already know.

  8. Margo/Mom says:

    Dan:

    Be that as it may regarding assumptions made by individual communicators, journalists, etc. It gets a bit dicey when we have to select one right set of cultural icons to include in education for all students. There is no culturally neutral way in which to do this. Nor can I perceive any particular advantage to setting a commnon curriculum at the level of who reads what authors and when.

    It is far more important (in my opinion) to experience good literature, than to toe the line with regard to each of the particular authors that may be included in even a preponderance of scholars recommendations (if there in fact were such a thing). Might someone, perhaps, who had extensive experience in the Andes, bring with them a good bit of the experience requisite to comprehend a passage set in the Appalachians? Many of the kids who are “culturally excluded” from the “content” do in fact have plenty of life experiences. They also have many educational deficits that have been documented repeatedly–less qualified teachers and on down the line.

  9. John Drake says:

    [Margo/Mom]: Blah, blah, leftist assumptions, cultural relativism, blah, blah, evil Western civilization, blah, blah, street-learning, blah, blah, scare quotes, blah, blah. [/Margo/Mom]

  10. Hirsch’s recommendation is spot on! It is a much more authentic task to teach students to dissect text in content that is one of the goals of learning. Reading comprehension is a complex process that involves a variety of strategies working together simultaneously, including the use of background knowledge. The strengthening of that background knowledge ought to be a goal of that instruction, and therefore an assessed item.

    Since we study Ancient Civilizations in 6th grade, it would be a fair measure of the students’ reading comprehension to give them an excerpt from a historic or contemporary piece related to the very history that they have just studied? If their background knowledge has been bolstered by their study, they will be able to comprehend with greater ease the text in question.

    Both aims are legitimate goals that will improve our students ability to understand text in the current context and novel contexts in the future.

  11. I did some bias screening for the state test not too long ago. It is a heck of a task to write this stuff for the full range of students in the state (urban centers, suburbs, ex-urbs, and extremely rural). One of the things I learned while collaborating with teachers around the state is how little the old test reflected the reality of those teachers out in places so rural their student load was 50 and they taught all of English 7 – 12. It was quite literary and matched up perfectly with our suburban curriclums. Now, I happen to think the new test is less demanding, but it is more general and non-fiction based — more of a pure reading test as opposed to literary analysis — and more closely reflects the rural majority.

    I’m not sure a curriculum should look exactly the same for a rural, agricultural district and an suburb full of university professors’ children.

    To take it one step further, I even change up my reading list from year to year depending on a) what kind of group I have and b) mood -keeping myself fresh trying new things or rotating texts (even my beloved Jane Eyre gets a little tiresome after teaching it three times a day for 10 years).

    On my third, ambidextrous hand we do have kids who transfer in out of sequence. We do Am Lit junior year while other districts do it for sophomores.

    I wouldn’t want to see a set reading list, but maybe a flexible list of types of texts would work (although I’d just say NO to that committee) — our state guidelines on skills are already pretty clear and in a decent sequence.

  12. Margo/Mom says:

    John:

    Actually, I was thinking more about the right wingers who go bullistic if kids are required to read Catcher in the Rye, or the picture story about the two male penguins, or anything that makes reference to the race of the person who assassinated Dr. Martin Luther King, Jr.

    But, I think LS has it right. There are many factors that go into the choice regarding what students will read. (hopefully based on LS criteria, not as one of my kids’ English teachers told me–he had to look and see what was “available” after the other teachers had made their selections).

  13. “I’m not sure a curriculum should look exactly the same for a rural, agricultural district and an suburb full of university professors’ children”

    I live in a rural ag district and I want my kids to have the same rigorous curriculum as the professor’s kids. The fact that we chose a rural lifestyle does not mean that my children are less capable than the suburban kids, just that we chose a different lifestyle. How in the world can my kids compete at university with the professor kids if they don’t have the same curriculum?

    Jane

  14. Andy Freeman says:

    > The basic idea is that readers use “background knowledge” to access new information. Harry Potter is a novel filled with archetypal images that the kids recognize from books, movies, and TV shows with which they are familiar.

    So, you’re arguing that Wizard land is more familiar than “the country”.

    Does that apply to fiction about “not city” or just reality?

    Can NYC kids understand stories about LA? How about Mexico City? Are the Jets unable to understand stories about the Sharks?

    I got it right the first time. These kids can’t understand what they read. They can only look for cues to things that they know from other sources.

    Shouldn’t we do something about that?

  15. Jane, I’ve never seen education as a competition. I realize there’s much hype about who gets into what schools, GPA’s, etc., but if first prize is knowledge, then it is not a zero-sum game with one winner.

    I’m not saying all curriculums shouldn’t qualify students for college; I’m saying perhaps it isn’t Great Expectations for everyone. Perhaps My Antonia would be better for certain populations. Don’t know. Just noting the general disatisfaction of my peers in rural districts when we suburban teachers seem to be in control of standards.

  16. SuperSub says:

    There are two different skills here. Hirsch promotes analysis based upon prior knowledge while reading passages based upon random content do in fact test a student’s ability to interpret new information.
    I’d say that both are valuable skills… but teachers and test-makers should be aware of the difference and carefully design reading passages to meet the goals of the assessment.
    In a content-based exam, the reading passage should be grounded in prior knowledge, while an exam focusing on reading ability should be more random in its selection of passages.

  17. With Harry Potter JK Rowling choose a hero who is new to the wizarding world as much as her readers so she could introduce her new world through his eyes. Not all nonfiction writers are as easily accessible. Surely education should aim at something slightly more demanding than merely equipping kids to read popular fiction? I have nothing against pop fic per say but there are writers without those skills who still sometimes are valuable to try to understand.

  18. Andy Freeman says:

    > With Harry Potter JK Rowling choose a hero who is new to the wizarding world as much as her readers so she could introduce her new world through his eyes.

    I’ll quote the objection: “Children are asked to read and then answer multiple-choice questions about such topics as taking a hike in the Appalachians even though they’ve never left the sidewalks of New York, nor studied the Appalachians in school.”

    There’s nothing in there to suggest that it would have been acceptable to ask the kids questions about a text set in the Appalachians if said text was written from the point of view of an NYC kid in the Appalachians.

    That objection says that it’s unreasonable to ask NYC kids about text dealing with experiences that they haven’t had. It says that it’s unreasonable to expect them to be able to discuss things from the text, that they can only address things from their non-text experience.

    If that’s true, their reading education is deficient.

  19. As I recall my reading comprehension tests, the passages were about places that NONE of us had heard of or been. (I grew up in a middle-class suburb, not near a major metropolitan area.)

    So, if the subject of the passage is not familiar to ANY of the students takint the test, is not the test perfectly unbiased?

  20. Andy,

    I relate to your assertions, yet I am again focused on the importance of background knowledge. Check out Joanne’s link to the core knowledge blog for more evidence that it’s the previous knowledge we have strongly influences are ability to relate to and understand new information.

    http://www.coreknowledge.org/blog/2009/03/24/understanding-is-remembering-in-disguise/

    The sort of discussion this leads to – the integral nature of vertical teaming and exit standards in a curriculum – is imperative to any education reform and true assessment of reading skills.

    For me, this epiphany came when I was a graduate student and took my first socio-linguistics class. My reading skills are generally excellent, yet I was finding an inability to understand the text. The teacher clearly noted I was unprepared to read that information. Without effective background and training in that genre, my basic skill was minimized.

  21. SuperSub says:

    What is reading but the effective use of background knowledge? Students at any level must know how to use grammar and definitions to understand the passage.
    Saying that a student in the city can’t answer questions dealing with the Adirondacks simply means that the teachers have not adequately prepared the students for the exam – the students have no chance but to rely upon background knowledge that they received outside of school.

  22. Andy Freeman says:

    > For me, this epiphany came when I was a graduate student and took my first socio-linguistics class. My reading skills are generally excellent, yet I was finding an inability to understand the text.

    So, any piece about the Appalachians is necessarily like a jargon and technical-term laden socio-liguistics text.

    I’m not buying it.

    These kids can’t read as it is conventionally understood. They can just pick out words that remind them of things that they learned elsewhere.

  23. David Cohen says:

    We seem to be talking about students in general a bit too much. I think Andy is talking about students with the lowest skills, and almost any decent test will show those struggles. Likewise, there are the highly skilled readers who bring not only decoding and vocabulary to the test, but also a set of skills and attitudes that give them confidence in their ability to relate to all sorts of texts in some way. The expect reading to make sense and are comfortable being drawn out of the limits of personal experience. They are also likely to believe that the test is a positive or neutral device, and to trust in its fairness.

    But I think there’s a large group in the middle. For them, the tests present some discomfort. They may feel that reading in general is a trap – a way for teachers and schools to catch you not knowing something. They expect reading to be a chore rather than a pleasure, and they are quick to shut down or tune out because they have less experience in pushing through and finding reading worthwhile. There problems are exacerbated by a distrust of schools, teachers, and tests. I would expect that this is a sizable group of students.

    So, if students arrive with decent skills but an uneasy mindset about reading and testing, the question we face is what to do to best measure their skills; offer familiar topics to engage the student as test-taker, or offer unfamiliar text which will appear more challenging even if the linguistic traits are similar?

    The best suggestion above was to try to capture both in testing, since both are valuable. If reading passages on tests cover familiar ground, we’re more likely to capture a picture of the students at their best. Still, it’s worth measuring how students perform with more challenging material, since our long-term goals include the skills to handle that kind of reading.

  24. David,

    Very insightful. That is a clear and pragmatic synthesis of the two ends of this discussion. Well said.

  25. Andy Freeman says:

    > They may feel that reading in general is a trap – a way for teachers and schools to catch you not knowing something.

    Welcome to life.

    If we only test on texts such that students can answer from other sources, we don’t know if they’re answering from their understanding of the text or from their other sources.

    I think that students should be able to understand from the text and that it’s a problem if they can’t.

    David doesn’t think that we should be able to detect whether they can do so. It’s unclear how we can help the ones who can’t if we can’t detect them.

  26. This reminds me of a blurb my 9th grade social studies teacher had the class read. It was a story about another culture and their racs. The whole point was to make the culture appear nonsensical. Of course the punch line was to spell rac backwards, which changed the whole meaning of the story and the arguments began.

    Another example is in a book written by Robert Burton with the main title “On Being Certain”. He does a simple experiment on the reader by presenting a short blurb that at first seems completely disconnected. At the end of the story he mentions a single word, the result is interesting.

  27. Andy, sorry about the delay in replying. I was travelling and for some reason my cellphone suddenly refused to download the comments on this post.

    I think you misunderstood my point. I am not arguing that kids can only answer questions from their non-text experience, clearly kids can learn some things entirely from reading about them. I was however saying that J. K. Rowling is very skilled at how she introduces her readers to her wizarding world, and pointing out the particular device (certainly not unique to J. K. Rowling) of using a narrator who is as new to her fictional world as the readers are. The fact that kids read and enjoy her books en masse does not mean that there is no point in teaching background knowledge – not all authors are J. K. Rowling.

    A text that writes about the Appalachians from the point of view of a NYC kid in the Appalachians may be a good way of introducing NYC kids to the Appalachians. BUT there are several problems with this approach generically:
    1. This depends on the writer’s skill level. Merely putting in a NYC kid as the narrator still leaves a lot of opportunities to write badly. J. K. Rowling is world-class at what she does, we cannot expect every writer to be as good as her.
    2. How about if the writer has some important things to say about the Appalachians but doesn’t know any typical NYC kids? Or isn’t that good at psychology in the sense of putting themselves in someone else’s mental space?
    3. How would this work with kids who come from areas that are unlike both the Appalachians and NYC?

    That objection says that it’s unreasonable to ask NYC kids about text dealing with experiences that they haven’t had. It says that it’s unreasonable to expect them to be able to discuss things from the text, that they can only address things from their non-text experience.

    Sorry Andy, you’ve misunderstood the argument. The objection is that it is unreasonable to compare test answers where groups of kids have very different background knowledge. Yes, kids can answer questions from only texts if the texts are well-written in the first place, but the argument is that, all other things being equal, readers can answer questions far better if they have relevant non-text experience (not necessarily real-life experience, a person can learn a lot about a sport say from watching it on TV, or from following it via the newspapers). It’s not that it’s impossible to learn anything from merely text alone, just that it is a bit more difficult. And the less-world-class the author, the harder it gets for the reader.