Thinking and Linking by Joanne Jacobs
Mismanagement, waste and shoddy construction plague the Los Angeles Community College District’s $5.7 billion rebuilding program, reports the LA Times.
Also on Community College Spotlight: Facing $400 million in budget cuts, California community colleges may stop subsidizing classes for students who aren’t moving toward a degree and “activity” classes such as yoga, dancing and drawing.
Who’s right about the value of value-added? A University of Colorado analysis challenges the validity of the Los Angeles Times’ value-added analysis of teachers’ effectiveness. The Colorado “critique is more cautionary than damning,” argues Rick Hess.
. . . this is a case where I think the results mostly highlight the import of moving carefully and thoughtfully on value-added. That said, the standard in crafting value-added systems ought not be perfection, because nobody anywhere in the private or public sector has got a system that can meet the standard. The question is whether a given system is better than the alternative. And the truth is that today’s personnel systems are so insensitive to performance, so protective of mediocrity, and so dismissive of excellence, that value-added systems need not be flawless to be good and useful tools.
Washington D.C.’s teacher evaluation data will be used to assess principals, teaching coaches and education schools, reports the Washington Post.
Now in its second year, IMPACT uses five classroom observations to rate how effective a teacher is in nine standards — including explaining content clearly and engaging students — deemed essential to good teaching. Certain teachers are also judged on whether their students’ test scores sufficiently improve — a metric known as “value-added.” All of the numbers are crunched into a teacher’s annual rating, ranging from ineffective to highly effective.
Last year, former D.C. schools chancellor Michelle A. Rhee fired 75 teachers with poor IMPACT evaluations and gave bonuses to more than 600 top scorers.
In the future, D.C. will use the data to determine which education schools are producing high- or low-scoring teachers, said Jason Kamras, the district’s personnel chief. “We’ll just stop taking graduates from institutions that aren’t producing effective teachers.”
Just as teachers are being held accountable for students’ performance on tests, Kamras said, administrators will be held accountable for teachers’ performance on IMPACT evaluations. Teacher ratings from one cluster of schools might be compared with those from another cluster to assess how a particular instructional superintendent is faring. Principals will be judged in part by the number of “highly effective” teachers they are able to retain from year to year. Instructional coaches will be held accountable for the ratings of the teachers they coach.
IMPACT also will help the district target teacher training to areas of high need, Kamras said.
IMPACT is too flawed to be reliable, said Nathan Saunders, president of the Washington Teachers’ Union. He worries that use of IMPACT scores will lead D.C. to stop hiring teachers from historically black colleges and universities.
The Los Angeles Times ranked teachers by value-added scores based on “demonstrably inadequate” research, concludes a National Education Policy Center study, Due Diligence and the Evaluation of Teachers. University of Colorado researchers found “serious weaknesses” in the LA Times’ work.
. . . it is likely that there are a significant number of false positives (teachers rated as effective who are really average), and false negatives (teachers rated as ineffective who are really average) in the L.A. Times’s rating system. Using the Times’s approach of including only teachers with 60 or more students, there was likely a misclassification of 22% (for reading) and 14% (for math).
But that’s not how the newspaper sees it. Separate study confirms many Los Angeles Times findings on teacher effectiveness is the Times’ Feb. 7 headline, claiming NEPC “confirms the broad conclusions of a Times’ analysis of teacher effectiveness in the Los Angeles Unified School District while raising concerns about the precision of the ratings.”
After re-analyzing the data using a somewhat different method, the Colorado researchers reached a similar general conclusion: Elementary school teachers vary widely in their ability to raise student scores on standardized tests, and that variation can be reliably estimated.
But they also said they found evidence of imprecision in the Times analysis that could lead to the misclassification of some teachers, especially among those whose performance was about average for the district.
NEPC is livid about the Feb. 7 story (pdf), saying the study by Derek Briggs and Ben Domingue “confirms very few of the Times’ conclusions.”
“We raised major concerns was with both the validity (“accuracy”) and reliability (“precision”), and our bigger focus was on the former rather than the latter,” writes Professor Briggs.
While the Times claims the study “largely confirmed” the newspaper’s rankings of most and least effective teachers, Briggs responds, “No, we did not, quite to the contrary. (Reporter Jason) Felch seems to be again focused only on the
precision issue and not on the accuracy problems that we primarily focus on in our report.”
According to LA Times reporters, many Los Angeles teachers are eager to get value-added feedback on their work, reports GOOD, which was part of a conference call with investigative reporter Jason Felch and Beth Shuster, the Times’ K-12 education editor.
A special education teacher, who wasn’t rated because she didn’t teach enough students, asked the Times to assess her and other special ed instructors, Shuster said.
When we opened up the database for teacher comments, before it went live on the website, we had a number of teachers who wrote into us requesting their private page—where they could see what their ratings are. And even before they got that information, they were asking us, “What more can you give us? Are you just going to give us this one number? Are you going to give us math and English broken out? How much more can you give me? I’m planning for the upcoming school year.” I mean, these people are asking a newspaper for this information. It just strikes me that these people are victims of the system. The district has not done anything to help these people. they’ve never gone in and helped these people in anyway, the good ones or the bad ones.
Principals at high-achieving schools were less likely to know which teachers were raising or lowering students’ test scores, Felch said.
. . . they’ve been under no pressure to improve, and the principals are not very focused on teacher quality. Because the kids come in at a very high level and score very high on achievement tests, they’re kind of resting on their laurels. … It was at those schools where we found a real disconnect between what the principal’s point of view was and what the data was telling us.
Parents need more than “parking lot chatter” to tell which teachers are helping students the most, Shuster said. If that causes chaos, with parents demanding their child get the highest scoring teachers, “That’s kind of a marketplace at work, isn’t it?” asks Shuster.
Los Angeles doesn’t reward, recognize or try to learn from its most effective teachers, reports the LA Times in a follow-up to its value-added analysis of third- through fifth-grade teachers’ effects on their students’ test scores.
The Times found that the 100 most effective teachers were scattered across the city, from Pacoima to Gardena, Woodland Hills to Bell. They varied widely in race, age, years of experience and education level. They taught students who were wealthy and poor, gifted and struggling.
In visits to several of their classrooms, reporters found their teaching styles and personalities to differ significantly. They were quiet and animated, smiling and stern. Some stuck to the basics, while others veered far from the district’s often-rigid curriculum. Those interviewed said repeatedly that being effective at raising students’ performance does not mean simply “teaching to the test,” as critics of value-added analysis say they fear.
On average, these teachers’ students improved by 12 percentile points on tests of English, from the 58th to the 70th, and 17 percentile points in math, from 58th to 75th, in a year.
Thomas Kane, a Harvard education researcher, tested the reliability of the value-added approach in Los Angeles, the Times reports. Kane predicted the student gains for 156 teachers who volunteered for the experiment.
Value-added analysis was a strong predictor of how much a teacher would help students improve on standardized tests. The approach also controlled well for differences among students, the study found.
With $45 million from the Bill and Melinda Gates Foundation, Kane and other researchers are now following 3,000 teachers in six school districts to see if other types of evaluation — including sophisticated classroom observations, surveys of teachers and reviews of student work — are also good measures of teacher performance.
In the meantime, Kane said that, although it is not perfect, “there is currently not a better measure of teacher effectiveness than the value-added approach.”
Teachers on the Times’ most effective list said they’d never been recognized for excellence. Aldo Pinto, a 32-year-old teacher at Gridley Street Elementary School in San Fernando, said, ”The culture of the union is: Everyone is the same. You can’t single out anyone for doing badly. So as a result, we don’t point out the good either.”
Value-added is the worst form of teacher evaluation, but it’s better than everything else, writes Chad Aldeman on The Quick and the Ed.
Los Angeles Unified now plans to share value-added data with teachers privately and hopes to negotiate its use in teacher evaluations with the teachers’ union. Tennessee did just the opposite, Aldeman notes. “Every year since the mid-1990’s every single eligible teacher has received a report on their (value-added) results.”
When these results were first introduced, teachers were explicitly told their results would never be published in newspapers and that the data may be used in evaluations. In reality, they had never really been used in evaluations until the state passed a law last January requiring the data to make up 35 percent of a teacher’s evaluation. This bill, and 100% teacher support for the state’s Race to the Top application that included it, was a key reason the state won a $500 million grant in the first round.
While LA teachers are angry and confused, Tennessee teachers have had time to understand how value-added analysis works and prepare to accept it.
The Los Angeles has posted its list of the most and least effective third-, fourth- and fifth-grade teachers and schools with teachers’ comments. A few teachers are challenging the data, saying that they’re listed as teaching in years when they were on leave. The value-added analysis of schools is interesting.
On City Journal, dducation researcher Marcus Winters looks at the pros and cons of value-added analysis and comes out against release individual teachers’ scores.
Test-score analysis is “correct” on average—it can tell us a great deal about aggregate teacher quality. It can also help to evaluate individual teachers. But given its messiness—especially when tied to stakes as high as people’s jobs—it cannot be used in isolation.
Critics go too far, however, when they claim that these limitations justify abandoning the value-added approach altogether. The real lesson is that test scores are best used to raise red flags about a teacher’s objective performance; rigorous subjective assessment should follow, to ensure that the teacher is truly performing poorly. If both analyses show that a teacher is ineffective, then action should be taken, including removal from the classroom.
Economic Policy Institute also sees Problems with the Use of Student Test Scores to Evaluate Teachers.
Despite laws preventing the use of test scores in teacher evaluations, California, New York and Wisconsin are trying to argue that they qualify for Race to the Top funding, reports Ed Week’s Stephen Sawchuck. He summarizes:
New York: “OK, our law says you can’t use test data in teacher-tenure decisions, but teachers have to demonstrate how they’ll use data to get tenure. Besides, the law only refers to tenure, not all those other teacher things.”
California: “OK, just because there’s a state prohibition on the use of this data doesn’t mean local districts can’t choose to include it on their own. Like, six whole districts already do!”
Wisconsin: “OK, we can’t use our NCLB tests for these teacher-related purposes, but we have all kinds of other tests we could use!”
California Superintendent Jack O’Connell visited Long Beach Unified, known as a data-driven district. The LA Times reports:
Seven years ago, the district developed a sophisticated centralized data system that allows it to track individual student achievement, attendance and discipline over time. The system also lets the district see how students are faring collectively in a particular classroom or school, and how subsets such as English learners or special education students are performing. District officials can then use the information for staffing decisions, such as where to send specialists.
Tom Malkus, principal of Lee Elementary School, said he and other school leaders use the data to spot struggling teachers and offer coaching, professional development and other support.
If that fails, (Superintendent Christopher) Steinhauser said, the district has “courageous conversations” with teachers that can result in their leaving the profession.
California Gov. Arnold Schwarzenegger wants to terminate the data firewall. The unions should agree to change the law, editorializes the San Jose Mercury News.
Without linking student data to teachers, lawmakers will shoot in the dark when they try, for example, to make sure that effective teachers are working in low-performing schools.
Swift & Change Able looks at New York state’s data firewall, which reads: “The teacher shall not be granted or denied tenure based on student performance data.”
Here’s how it would have been written if what NY state officials and the unions are saying was their real intent really was their intent (the simple adding of one word):
“the teacher shall not be granted or denied tenure based solely on student performance data.”
or, more elegantly and affirmatively:
“teacher tenure shall be granted based on student performance data and other relevant factors”
. . . The one upside of this debate: we now know what is meant by “creative problem solving” when union officials and their flacks talk about “21st Century Skills.”
On Flypaper, Mike Petrilli looks at the politics of the decision to require states to use achievement data to evaluate teachers. It’s not just a poke in the eye for the teachers’ unions, he writes. It’s a poke at California. And it couldn’t have happened without the OK of Rep. George Miller, a liberal California Democrat who chairs the House Education and Labor committee. Petrilli thinks it’s Miller’s revenge on the NEA, which made “a stink about merit pay when Miller’s NCLB reauthorization bill was floated back in 2007.”
Seven years after Los Angeles school officials fired a special education teacher for sexual harassment, a judge has approved the firing of Matthew Kim, reports the LA Times. Kim “was accused of touching co-workers’ breasts and making improper advances toward students.”
All told, the Los Angeles Unified School District has spent nearly $2 million, including Kim’s pay and benefits while he was barred from the classroom.
Known as a “housed” employee, he and about 160 others reported every day to administrative offices, where they were assigned no work.
The state Commission on Professional Competence “agreed that some of Kim’s actions could have been considered sexual harassment but ruled that he should not be fired.” The district won the case on appeal.
The Times wrote about the long, costly process of firing teachers, including a story on “housed” teachers that spotlighted Kim’s case. The teacher’s response is here.
Compton middle school students went on a field trip to Bear Stearns to learn about careers in finance. But they didn’t know enough to understand what they saw at the investment firm, a foundation official told Sandy Banks, an LA Times columnist.
“We’re trying to teach them about portfolios and they can’t even spell the word, never heard of it!” Veronica Coffield told me in a voice shot through with urgency. “They’re still learning ‘less than’ and ‘greater than’ in eighth grade, and they’re supposed to make it through high school?”
Singer Chaka Khan’s foundation sponsors a “Going to College” program for students at low-income, all-minority Drew Middle School. Originally, it was all about field trips: “The preteens learned about the justice system in the television courtroom of Judge Judy, about health and fitness in Tae-Bo classes with Billy Blanks, about culinary careers at restaurants in Malibu and Beverly Hills.” But Coffield realized students lacked basic reading, writing and math skills.
“We’ve got eighth-graders with an A in algebra who can’t tell me what six-times-five is equal to!” Coffield said. “Seventh-graders who don’t know the difference between a noun and a verb!”
. . . (Students) described math classes crammed with unruly students, some of whom could barely add and subtract; an English class with no permanent teacher but a succession of unprepared subs; teachers who ridiculed wrong answers in class and swore at students in the halls.
But it wasn’t as simple as poor schools or bad teachers. Students cut class and ignored assignments. At the project’s orientation meeting, one mother strode in cursing loudly, high on drugs.
The foundation recruited volunteer tutors from USC. But five of the 36 eighth graders in the program failed too many classes to graduate. They didn’t attend the graduation ceremony, but they’ll all go on to high school in the fall.
“They can’t stay here because there’s nowhere to house them,” a Drew teacher explained.
Not “educate,” Banks points out. “House” as in warehouse.
Recent Comments