The expectations gap

Eduwonk links to Education Trust slides of seventh-grade writing assignments at two California middle schools. One asks students to analyze Anne Frank’s character; the other asks students to write about “my best friend” or “a chore I hate.” The expectations gap is huge, he writes.

Raising expectations doesn’t work unless other things happen too, writes commenter John Thompson, a high school teacher in Oklahoma City.

To turn things around, it would take an honest discussion of what it takes to raise student performance when neighborhood secondary students have skills that are five years or so below their neighbors who went to magnet schools. But our district leaders had the the same visceral response as you seem to be having, and mandated immediate and much much higher standards. Instantly, many core teachers were intimidated into teaching five years above the students reading level, and failure rates soared to 95% in some. The dropout rate exploded and the district immediately abandoned the experiment.

Look at what’s being taught, adds Core Knowledge.

You can’t feed kids a thin gruel of content-free, “self-directed” reading and writing for their entire academic career and then expect them to suddenly be able to write a nuanced character study of Anne Frank in the 7th grade.  You can’t ask kids to do “self-directed” writing about their family, their friends and their personal experiences throughout elementary school to the exclusion of nearly all else, then expect them to dazzle you with their insights into literature in middle school.

Education Trust’s new report details an old complaint: The neediest students are the most likely to get teachers who didn’t specialize in the subject they’re trying to teach. It’s especially hard to hire and retain qualified math and science teachers in low-income, high-minority middle schools.

About Joanne


  1. I find that few people look at the raw, absolute data. They just look at transformed, relative statistics. They don’t really know what they are trying to fix so schools use their favorite method of problem solving; guess and check. If the numbers go up, they are happy. If they go down, then they just guess at a solution. This happens in high SES communities too, except that very low state standards make it seem like they are doing such a great job.

    This isn’t just an issue of urban versus suburban schools and the training of the teachers. This isn’t just a problem you solve with money. My son gets silly writing and art assignments all of the time. In sixth grade, he had to draw a picture of his favorite scene in a book, rather than write a book report. It’s just that many parents make sure that real learning takes place.

    If there is something I’ve learned over 35+ years of programming, it’s that you need to define exactly what the error is and work backwards. You don’t correct problems with statistical guess and check. When I used to teach college math and computer science, I had students who always tried to debug a program correct; they kept trying things until the error went away. Unfortunately, they didn’t know why the problem went away, it might still be there, and it might have caused other problems.

    Our school has parent-teacher meetings to review our state test results. They look at a category, see if the results are going up or down, and then guess at how to fix the problem. They don’t look at the actual questions on the test and ask themselves why students don’t know that material. It’s too scary to do that. They would have to confront the fact that these are trivial questions, or that they just don’t teach that material.

    Schools have to get their heads out of the relative statistical sand and look at the raw data. They have to look at the actual questions and examine case studies of exactly why some kids do well and some don’t. The study of why some kids do well will probably be more informative than the study of why some kids do poorly. Our state sends out a questionnaire to all parents, but there isn’t a single question about what parents do to make sure that learning gets done.

    If kids get to fifth grade and don’t know how to tie their shoes, then that tells you something real. This is hidden if all you look at is a transformed, relative “proficiency index”. In our state, the horrible, raw percent-correct scores are transformed into a proficiency index (based on a very low cut-off score) to create a percentage that looks a whole lot better (between 80% – 100%). If you look at the proficiency index, things don’t look so bad, and you will focus on relative improvement. If you look at the actual test questions and raw results, you will be horrified.

  2. Lightly Seasoned says:

    That would be a neat trick — looking at the test and seeing which children answered which questions correctly or incorrectly.

    The test booklets are strictly accounted for and are sent back at the end of the testing window. A missing booklet is a big deal. Some questions are released every year, but we certainly don’t get a printout of how the kids did question by question. Making copies of the book or recording the questions in some way is cheating.

    We do get an analysis of how they did on certain skills, but the skills are tested in multiple mysterious ways in constructed responses, so we really don’t know what to do with some of the results. We know some general trends to work on, but the data is useless in the way you would propose we use it.

    We are getting a brand new type of test this year, and we’ll see if it does a better job giving us information about what the kids know and don’t.

    The AP tests give me some useful information on their reports (ie. if the kids scored poorly on the Q2 I know I have to work on that with the next group), but not a question-by-question analysis. I would like a report from the state that is a big more like the CB’s. When I go out to write the test questions in a few months, I’ll be lobbying for useful test analysis.

  3. “Some questions are released every year, but we certainly don’t get a printout of how the kids did question by question.”

    That may be true for standardized tests, but there is nothing stopping schools from doing their own analysis and case studies locally. It’s almost as if schools have lost all knowledge of formative assessment and accountability. They just turn to the state and wait for them to provide the data. Schools can’t let the state’s minimum proficiency cut-off become the maximum target. Set higher goals and the standardized tests will take care of themselves.

    I agree that standardized tests hide the raw data. They use magic methods for converting the results into statistics. But you can still look at the released questions and the raw percent correct scores. In working with my school, many never looked at actual questions. They only looked at relative results in the magic categories. They might say something like: “We need to work on problem solving.”, even though they have no direct idea how changes will affect results. States really have to provide ALL of the questions and the raw percent-correct numbers for each school, grade, and subject.

    This reminds me of the rubric grades my son gets. Rather than give him direct corrections and comments, he gets results encoded into a vague list of goals. They expect more from him, now that he is in 7th grade, but that includes reading the teacher’s mind.

  4. Every teacher I’ve talked with has said that the results from standardized tests are useless as far as helping students and helping them to improve teaching skills. I have heard from school administrators that standardized tests have helped teachers focus on a given curriculum so that test scores do improve. I haven’t heard teachers mention this aspect in a positive light, they usually call it teaching to the test. This experience isn’t surprising as it wouldn’t be easy to tell a parent that the standardized tests had actually found something a teachers was forgetting to teach. And as far as standardized tests narrow the curriculum I’m not a fan of teaching to the test either.

    These responses make some intuitive sense to me as the standardized tests come only once a year, at least in CA. That’s very slow feedback. I think another part of the reason teachers find standardized test results useless, at least in CA, is that they are already doing periodic evaluation of students, at least every teacher who has taught my children. So teachers already know what students are struggling with both on an individual basis and on a collective basis. With the direct evaluation teachers have the raw data and can act on it immediately. Its seems a lot more effective to focus on improving the direct teaching experiences than on standardized test. Don’t get me wrong, standardized tests can still be useful to catch the gross oversights.

  5. Lightly Seasoned says:

    SteveH: Right. That’s how I design MY assessments. To what degree they align with the state assessment is fuzzy because I don’t know how they’re assessing certain things. For example, grammar is assessed two ways: some multiple choice questions and in the constructed responses. These are actually two different skills (editing and composition) being muddied. Are my kids doing poorly on editing or in their written responses? Who knows?

    Actually, probably the editing because I do very little of it — I’m teaching students who need to write well, not edit other peoples’ writing. Which would you rather have your child learn how to do? The two skills don’t transfer very well. I also spend a huge amount of time teaching research skills, which the colleges and universities are screaming for. Our state test doesn’t test them — at all. Yet, our graduates state repeatedly that they were very well prepared for college because of that emphasis.

    Standardized testing can help focus a curriculum — my AP curriculum is HIGHLY focused — and I’m a better teacher for it. BUT, what’s being tested has to be clear and meaningful. And it has to have a logical progression. You can’t leap from personal essays to literary analysis; you have to bridge the two over many assignments. Personal essays are often favored in low performing classrooms because it is a way to get some fluency in composition without running into the brick wall of reading comprehension — it separates the two skills.

  6. When I look back at my education and at my daughter’s problems in teaching an 8th grade AP class the form and the relationships stand out. A classroom has the advantages and disadvantages that a city has as opposed to a small town for an adult. You blend into the mass. The mass and the teacher first of all establish a power relationship. Where this is not too dysfunctional, the teacher can establish a training schedule that the students can learn by. I think for most of us a useful next step, for students and teachers, would have been to work with someone that offered more personal review of the student’s opportunities and problems. Currently this is only a part of ‘extracurricular’ activities.

  7. There is no such thing as an 8th grade AP class.

  8. ” my AP curriculum is HIGHLY focused”

    Unfortunately, there is nothing like AP in the lower grades to keep schools honest. All we have are vaguely-defined and very low state expectations, and many can’t even meet those standards. It’s not because they don’t have enough feedback from state testing. They know where the problems are. They just don’t want to do what they know they have to do. They don’t want to separate kids by ability or willingness. The environment is about nurturing, not academics. They assume that all kids will reach their potential.

    But kids somehow have to transition from a low expectation, everyone is equal, K-8 educational philosophy, to a tracking high school, high AP standards, and a sink or swim philosophy. Our K-8 school is finally realizing that they can’t treat all kids as equals and then just toss them into high school, so they have turned up the pressure in 7th and 8th grades. However, my son still does art work instead of writing at times. Even when they set higher academic standards, the kids are not properly prepared. It’s easy to ask kids to analyze Anne Frank’s character, but it’s not so easy to prepare them to do so.

    My son’s social studies teacher had kids read articles about the presidential and vice-presidential candidates and write analyses about them. In my meeting with her, she talked about scaffolding, but she was talking about the process and not scaffolding of background knowledge. The articles talked about all sorts of things that none of the kids would know, but they had to answer questions about the strengths or weaknesses of the candidates, all in one or two paragraphs. Her scaffolding didn’t include how to condense a complex subject into one or two paragraphs. Kids struggled, and they assumed it was their own fault.

    The only areas where there has been some real change in our school are math and foreign language. This is only because the target in high school is well-defined; geometry for those heading for the AP track, and Spanish II for those who don’t want to start over in high school. These changes, however, only came about because parents screamed that their kids weren’t prepared properly. The gap was clearly defined. I don’t see anything like that for other classes.

    It’s not just a matter of providing content specialists. Our state requires this for seventh grade and above. It doesn’t fix the problem in urban schools. It has to do with a general philosophy of low academic expectations and full-inclusion in K-8. If you wait long enough, all problems look like they belong to the student. They will point to someone else’s amazing analysis of Anne Frank’s character. They will point to others who easily made it to the AP calculus track, but they won’t try to find out if there is something else going on.

  9. Margo/Mom says:

    Steve H:

    I don’t know where you just landed from–but you speak truth with good sense. LS–I don’t know all the specifics of how your state handles assessments, but in my state, I hear the same kinds of blaming, and yet, I know that while every question cannot be made available for every student, what is available comes very close–to the level of every question that is available (some are always withheld for future use) is available for every student. Even more important, there is aggregate information, such as the percentage of students choosing each of the wrong answers. I spent a good bit of time with one of my child’s principals on why they weren’t doing exactly the kinds of things that Steve advocates (and reporting this to parents–as NCLB requires). One thing that she provided me with was a copy of the test analysis, which provided question by question analysis at the building level. I understand that in many states/districts the difficulty in getting classroom level information is that unions are afraid that if this were disclosed, it might be used to evaluate teachers–which they oppose.

    But, as Steve points out–the standardized tests ought not be the only source of information. They are good for some things and not for others. They are very good at the macro level to point out areas of strength/weakness, as well as to draw correlations between some of the systemic problems about which we are always in denial (the impact of teacher experience/certification, etc and its maldistribution by SES, for instance). One thing that leaped out at me when I looked at the school report that I saw was that very few kids were getting anywhere on essay questions–and many did not attempt them. Personally, as a parent, I would far rather see an analysis, by staff, of areas that they saw as weakness with a plan for shoring them up than the sort of “give it the old college try, buck up,” attitude that I usually get–assuring me that all is well, teachers are working very hard, and pay no attention to that man behind the curtain.

    Teaching ought to be guided by the State Content Standards, with the test as an annual indicator. It would appear that one place that this always falls down is in moving from standards to curriculum. In my state, a good bit is left to the individual district–because the legislature and the voters prefer it that way. So the result is standards that are fairly general, so that they can be met in a variety of ways–districts get to pick their own texts, develop their own reading lists, etc. I am not sure that this works well–but teachers tend to be pretty hostile to anyone telling them what/how they are to teach. The flip side is that not every teacher arrives in the classroom ready to develop their own curriculum, principals are not prepared to evaluate the content of every classroom and mediocrity prevails. Districts respond by laying in place “pacing guides” to ensure that everything is “taught.” At the building level, wise teaching staffs can organize within this framework to collaboratively develop effective lessons, but I must stay, I haven’t seen a lot of this happening.

    I see resentment of what is required and what is not–whether it is standards, or curriculum or developing collaborative learning communities. I see middle and high schools who believe that their problem is that the lower grades produced students who are five years behind, but can’t look upstream to see why that is happening. BTW–this is a problem that was clearly NOT created by the tests. Before there were tests, the upper grades were happy to teach students who were whole grade levels behind. Now that we know that this was the case, they are mad that they are supposed to do something about it.

    I just finished reading about an inner city school where none of the third graders (there were about 40 of them) in their two classrooms had achieved reading proficiency. Notice–the average class size would be about 20. It is no accident, given the structure of the institution of education, that both teachers were first year teachers, or that one spent considerable time out on leave. The principal suggests that there was a higher than normal rate of absence and tardiness last year. Ya think? That was symptom number one. Symptom number two was the test scores. Any guess on what the disease is?

  10. Lightly Seasoned says:

    I don’t care if you believe me or not, Margo. Obviously, given the item about teachers being unable to express themselves online openly, I cannot divulge any information that could narrow down where I teach.

    FWIW, while we don’t get any information that pins down question-by-question scores, the scores are sorted by teacher. When we are able to pinpoint a particular skill that pops up pervasively, we absolutely address it.

  11. Kevin Smith says:

    I hear this “teaching to the test stuff a lot”. Must be in a state other than NC, we never see the test, just the curriculum giving broad areas to teach with more closely defined objectives under those areas. We don’t get individualized objectives. We do get high school kids who have to pass tests in five content areas in order to graduate that include three areas that were never tested until high school (civics, history, and biology). While the state has a Standard Course of Study for these areas in elementary school and middle grades there are no tests (so it is basically not taught). I’d like three more standardized tests in my state, in 8th grade, but our middle schools set the bar so low we end up fighting a war to convince the kids just to show up regularly (our middle school has no attendance policy ans social promotion is alive and well).

  12. Kevin,

    “While the state has a Standard Course of Study for these areas in elementary school and middle grades there are no tests (so it is basically not taught)”

    This is what I’ve always meant by “teach to the test”.