Similar schools, similar results

Most California schools perform about as well as expected over a three-year period when student characteristics are factored in, concludes a new similar schools measure developed by the California Charter Schools Association.  From California Watch:

Using a complex regression analysis, the measure takes a number of characteristics of the school’s student population into account. These include the socioeconomic background of the student body, the average education level of their parents, the number of students with disabilities, the percentage of English language learners, and the racial and ethnic makeup of the students.

Researchers say the measure is more accurate than the state’s “similar schools” ranking, which don’t include small schools and fluctuates from year to year.

Seventy-eight percent of schools performed within 5 percent of their predicted level on state tests, according to the CCSA’s analysis. Nearly 10 were just below and 10 percent just above the prediction.  That left only 2 percent of schools far below their level, and 1.6 percent far above.

California charter schools are four times more likely to be among the top 5 percent of schools that exceed their predicted test scores – and twice as likely to be among the bottom 5 percent across the state, said Samantha Olivieri,  CCSA’s accountability manager.

About Joanne

Comments

  1. Another limitation is that it places the majority of schools [PDF] – 78 percent – in a single category of performing within 5 percent of their predicted level on state tests.

    Why is that a limitation, as opposed to a conclusion?

    In other words, once you hold everything about the inputs constant, the outputs are the same. Or did I misunderstand?

  2. In other words, schools are like sewers:  what you get out of them depends on what you put into them.

    (apologies to Tom Lehrer)

  3. What it says to me is “most schools are doing essentially the same thing” and getting essentially the same results. Very few schools are trying anything that has a noticeable impact – and some of it works well, some of it doesn’t.

    I’m not surprised that charters are more likely to be outlier as they have more freedom to make changes. Analyses like this are a good tool to identify whether “new and improved” is actually an improvement.

  4. Schools have all kinds of different philosophies and methodologies, yet achieve broadly similar results with the same inputs.  What it says to me is that schools matter a lot less than the students.

  5. The predictions for all the schools were made usingstudent body characteristics (parent education level, socioeconomic status, etc.) so the outliers were the schools that had significantly different results (good or bad) than what would be explained by their particular students. In other words, the outliers were all about the schools.

    For the vast majority of schools in the middle of the predicted zone, yes, results were dependent on the student population.

  6. Don’t forget that there’s going to be statistical variation and some selection effects between schools, and they’ll be more prominent the smaller the schools are.