KIPP middle schools take as many transfer students as nearby district schools, according to a Mathematica working paper (pdf). Furthermore, attrition rates for black males are *lower* than in neighboring schools, Mathematica found.

“KIPP’s success is not simply a mirage that is based on the results of a select number of high achievers who persist through 8th grade,” the researchers write.

A 2010 study by Mathematica found large achievement gains at KIPP schools, even when the scores of students who had left the schools were included, Inside School Research notes.

A Western Michigan study found high attrition for KIPP’s black males, charging that 40 percent of black male students leave between sixth and eighth grade. The study compared two or three KIPP schools to entire school districts.

Mathematica compared individual KIPP schools to neighboring district schools. “Our data is showing that KIPP loses black males overall at a lower rate than the local district schools,” said Christina Clark Tuttle, a senior researcher.

Urban black male students often change schools, whether they attend a district or charter school, but are less likely to leave the district.

KIPP students are more likely to be black or Hispanic and have lower incomes than students in the surrounding school districts, Mathematica confirmed.

This keeps getting us back to the same old thing. If the issue is how well KIPP is implementing its model, the Mathematica methology and results are fine. KIPP does great things and its attrition rate is no worse than neighboring middle schools that are likely to be among the most dysfunctional institutions in America. If the issue is whether KIPP can be scaled up, Miron asks the better questions, and has the more relevent results.

I noticed that the OKC KIPP’s attrition data was excluded for methodological reasons. It is no criticism to say that the OKC KIPP, which had a decade to scale up to 220 students, and which has never come close to meeting its recruiting goals, does not teach the same kids as the old middle school that was cited in Harpers Index for a riot. That school of over 600 had a ONE YEAR MIDDLE SCHOOL DROPOUT RATE of over 10%! My middle school had a ONE-YEAR DROPOUT RATE that once approached 20%! Prorate those numbers over five or six years to see how many students neighboring schools lose. it wouldn’t take much for the KIPP (which admits to being disapointed by its high attrition rate) to top that of a school that had 30 EXPULSIONS. (do you know what it takes to EXPEL a middle school kid; routine felonies at school sure wouldn’t be enough)

Here’s the point, the Mathematica methodology could not approach the key question. Are the school-leavers moving horizontonally or upwards educationally, or downwards, perhaps in an extreme, tragic spiral? Until they try to address that, Mathematica scholars should do the jobs they were commssioned to do, and stay out of the broader debate.

The KIPP schools do not replace many of the students who leave*, leading to a huge loss in the size of the class cohort. Public schools do replace students who leave. That’s a confounding difference that means the Mathematica claims are misleading and invalid.

To give a clear picture: Let’s say 100 students start 5th grade at a KIPP school that serves grades 5-8. Sixty of the students leave along the way, before completing 8th grade, and those are the lowest-performing 60 students. They aren’t replaced with incoming students. So that cohort at the KIPP school winds up with only the 40 highest-performing students. (This is the real-life attrition percentage found in a 2008 study of KIPP’s Bay Area schools by SRI International**. SRI also found that it’s consistently the lowest-achieving students who leave.)

At the public school down the street, 100 students start. Sixty of them leave along the way, but each time one leaves, a new student arrives to replace him or her. So that cohort at the public school consists of 100 students from start to finish. Clearly, those two situations are not parallel, equivalent or comparable.

A close relative of mine is a researcher for RAND, the research organization. So I know that with all reports, the way the research is presented is is negotiated between the research organization and the funder — sometimes painfully and at great length. KIPP funded the Mathematica report. Draw your own conclusions.

*It’s not that KIPP has a POLICY of not replacing students who leave, but the numbers show decisively that a huge number of students leave and are not replaced. That’s not true at comparable public schools; students cycle in and out and the overall numbers stay the same.

**The response that “the San Francisco schools are outliers” Is false, misleading and invalid, so please don’t waste our time with it.

An additional point on KIPP schools: At least in the Bay Area KIPP schools, the 5th grades tend to be sparsely enrolled (oh yeah, those claims that “all KIPP schools have long waiting lists” are also false), and then show a bump in 6th grade — because the nearby elementary schools feeding the KIPP middle schools are K-5s.

Then the enrollment drops off sharply in 7th and 8th grades — that’s where the attrition makes itself felt.

The KIPP schools do not replace many of the students who leave*, leading to a huge loss in the size of the class cohort. Public schools do replace students who leave. That’s a confounding difference that means the Mathematica claims are misleading and invalid.You need to read the Mathematica report, because it specifically refutes your baseless claim about replacement rates.

Here’s what Mathematica says:

“Pooling the results across all offered middle school grades, there is no systematic pattern of differences in the prevalence of late arrivals at KIPP schools and comparison district schools. . . . Averaging the results across sites, the overall proportion of late arrivals at KIPP (15 percent) is similar to the overall

proportion in the district comparison group (14 percent). After testing for statistical significance in each site, we did not find any systematic pattern of differences in the proportion of late arrivals at KIPP, compared with district schools. Some KIPP

schools admit significantly more late arrivals, other schools admit significantly fewer late arrivals, and several schools are not statistically different from the comparison group.”

That’s weasely language that obscures the overall drop in the student cohorts at KIPP schools that have been studied. The fact remains. I’m focusing strictly on what’s revealed by the simple number of students enrolled.

For example: The movie “Waiting for ‘Superman'” presented the story of Daisy, who hoped to attend KIPP Los Angeles College Preparatory Academy and was devastated to lose out in the admission lottery. But in real life, the high attrition at KIPP LA Prep means Daisy shouldn’t have to wait long for an opening. For example, KIPP LA Prep’s most recent 8th-grade cohort lost a third of the students who started in grade 5 by the beginning of 8th grade – figures aren’t publicly available for how many students finished 8th grade.

For that class, which started grade 5 in the 2006-’07 school year and finished grade 8 in 2010, the number dropped from 97 students at the beginning of 5th grade to 81 by the beginning of 6th grade to 65 by the beginning of 8th grade.

So, weasel and make excuses all you want, KIPP defenders, but the fact is that that class dropped from 97 to 65 students, according to the California Department of Education website. That’s attrition and that’s what we’re talking about.

My two cents – I don’t care if they don’t replace kids; the ones that leave are giving up an opportunity and it’s their right to do so. The kids in a KIPP school are on average going to receive a better education than they would in a local school. As long as KIPP continues to provide a quality education to urban children who otherwise would be in a low-performing school I’m on their side. My only stake in this is that I’m an American that wants to see all kids educated as well as possible because it is good for my community.

Secondly, kids are not automatically brought into a classroom if a spot opens up – that only happens if a new kid happens to move into the district. My kids’ public school has very low enrollment even though we are in a very urban area due to changes in zoning. There’s plenty of room for more kids but they aren’t in the zone.

Caroline, I was thinking the same thing. There was no doubt that the number of KIPP students dropped dramatically in the Mathematica study. I remember us discussing it. So what is this study purporting to show?

John, I thought your points both here and at the other link were excellent.

Caroline — a simple-minded look at cohorts can’t refute a study that actually tracks what happens to individual students at KIPP schools vs. what happens to individual students at other public schools. You can stick your head in the sand all you want, but your favorite talking point just got trumped by the evidence.

By the way, no one questions that there is SOME attrition at KIPP schools, and that SOME of the attrited students aren’t replaced. But some are. And we now know that the overall pattern (contrary to Caroline’s specious and unsupported claims) isn’t any different from similar public schools, which also see SOME kids leave, SOME of which aren’t replaced.

Stuart,

I am puzzled. My experience is that, during a school year, some kids move into a district and enter a school while other kids move away and leave the school. On average, it all evens out. Generally, a school ends the year with about as many kids as it started (at least prior to high school, when kids begin dropping out).

Caroline seems to be saying that KIPP schools end 7th and 8th grade with considerably fewer students than they start that grade. But you seem to be reading the Mathematica report to say that, on average, they don’t lose any more than comparable district schools.

Is that because Caroline is looking at an unrepresentative subset? Do some KIPP schools actually end up with more students than they started (to balance out the KIPP schools that lose)? Do most middle schools end the year with fewer students than they started? And if so, where do they go?

Caroline is saying that in KIPP schools, the 8th grade class is smaller than you would have thought if 100% of the 5th graders from 3 years before had all stayed in the school. But she isn’t comparing KIPP to anything outside of her own say-so about what public schools supposedly do. This obviously isn’t the same as having actual data from actual inner-city public schools on how many students they lose and how many students transfer in. Nor does she have actual data on what happens to individual KIPP students, some of whom are held back (and aren’t quitting KIPP entirely).

When Mathematica looks at individual students, they find that on average, KIPP schools have similar or even less attrition than nearby public schools, and KIPP schools DO admit replacement students.

I took Caroline to be saying that in the KIPP schools she is familiar with, a 7th grade class that starts the year with, say, 100 students would generally end the year with something substantially less, say 90. The eighth grade class would do the same thing.

If this is true, then for KIPP schools to be comparable to other schools, all schools would have to lose a substantial number of seventh graders as the year goes by and also lose a substantial number of eighth graders as the year goes by.

But the only way for this to be possible would be for all schools to lose a substantial proportion of their students from the beginning to the end of seventh and eighth grade.

But then where do the students go? Special ed outplacement? What?

Other possibilities are that the KIPP schools she is familiar with are unusual or that something is wrong with the reported numbers.

Right now the math just doesn’t make sense to me. It seems to violate the law of conservation of students.

Well, as I understand it, Mathematica isn’t comparing KIPP to all public schools, just to those that are nearby. Perhaps, on net, there are students moving out of those inner-city districts over time, and there may be others that just drop out of school altogether.

Perhaps it’s just the Thomas Schelling in me (“The Inescapable Mathematics of Musical Chairs”) but that doesn’t seem right. I know big city schools are losing students over time, but this seems a lot faster than that. And to the extent that 7th and 8th graders are under 16, they can’t legally drop out. (Perhaps some of the recent drop is immigrants–legal and illegal–going home in response to a bad economy. But I’ve seen no good figures on that.)

Whatever the legality of dropping out might be, you have to remember what’s going on in the KIPP-type of neighborhood. I’m reminded of a passage from Michael Lewis’s NYT article that led to his book “The Blind Side.” He’s discussing a black kid from Memphis named Michael Oher:

In his first nine years of school, Michael Oher was enrolled in 11 different institutions, and that included a gap of 18 months, around age 10, when he apparently did not attend school at all. Either that or the public schools were so indifferent to his presence that they neglected to register it formally. Not that Oher actually showed up at the schools where he was enrolled. Even when he received credit for attending, he was sensationally absent: 46 days of a single term of his first-grade year, for instance. His first first-grade year, that is; Michael Oher repeated first grade. He repeated second grade, too. And yet the school system presented these early years as the most accomplished of his academic career. They claimed that right through the fourth grade he was performing at “grade level.” How could they know when, according to these transcripts, he hadn’t even attended the third grade?Simpson, who had spent 30-plus years in area public schools, including 29 in Memphis, knew what everyone who had even a brief brush with the Memphis public schools knew: they passed kids up to the next grade because they found it too much trouble to flunk them. They functioned as an assembly line churning out products never meant to be market-tested. At several schools, Michael Oher had been given F’s in reading his first term and C’s the second term, which allowed him to finish the school year with D’s — they were giving him grades just to get rid of him. And get rid of him they did: seldom did the child return to the school that passed him. The year before Simpson got his file, Michael Oher passed ninth grade at a high school called Westwood. According to his transcripts, he missed 50 days of school that year. Fifty days! At Briarcrest, the rule was that if a student misses 15 days of any class, he has to repeat the class no matter his grade. And yet Westwood had given Michael Oher just enough D’s to move him along. Even when you threw in the B in world geography, clearly a gift from the Westwood basketball coach who taught the class, the grade-point average the student would bring with him to Briarcrest began with a zero: 0.6.No doubt some drop out illegally, and some 8th graders may be 16 (though one of the points of Lewis’ story is that students get passed up to the next grade whether they deserve it or not because that’s the easiest thing for the school to do, so I wouldn’t think there are many 16-year-olds in middle school).

The question is whether there are enough people like this to make for substantial drop-offs in enrollment between the beginning and end of 7th and 8th grade.

It’s easy to see what’s really going on by looking at the data in the Mathematica KIPP report. Average attrition at KIPP is 6 times higher than in the public schools.

Look at Table II.2, which gives total enrollments in the KIPP and the district schools for 5 – 8th grade.

Raw Enrollment, 5, 6, 7, 8:

KIPP: 7750, 6453, 4365, 2683

District: 2696955, 2649411, 2667580, 2095341

The raw numbers for 8th grade are skewed because four KIPP schools had no 8th grade, and the public enrollments for these schools were also omitted from the totals. To account for this, take the 5th grade enrollment as 1 and calculate the relative enrollment for each subsequent grade. If we average these fractions for all 22 schools (omitting the 0 enrollments in 8th grade), we get:

Relative Enrollment, 5, 6, 7, 8

KIPP: 1.0, 0.82, 0.53, 0.36

District: 1.0, 0.95, 0.91, 0.89

KIPP schools clearly have a true attrition rate 6x higher (64%) than the public schools (11%)

It certainly looks like Mathematica is going out of its way to mask the true rate of attrition by ignoring the fact that individual students who leave the public schools are replaced by new students, while KIPP students are not replaced.