You are here: Home / Archives for value-added analysis

Some teachers break ranks on seniority

September 14, 2010 By

Some teachers are breaking with their union to support performance pay and oppose seniority, reports the Wall Street Journal.

As a teacher, Sydney Morris wants to be rewarded if she can show she helps students make progress in her classroom. She also wants to make job protections such as tenure more difficult to get, and in the event that layoffs have to happen she wants the worst teachers to be let go first, no matter how long they’ve been teaching.

In March, the Bronx teacher and a colleague, Evan Stone, started Educators 4 Excellence to mobilize teachers who want to change the status quo.

Of particular concern is the practice of laying off teachers based on how many years they’ve worked in the schools. That “provides a safety net to be complacent,” said Margie Crousillat, a member of Ms. Morris’s group who is a kindergarten teacher in the Bronx in her seventh year of teaching. “Some veteran teachers have been teaching 25 years and they are incredible. But some aren’t. I don’t think age or experience should dictate whether you’re safe in your job.”

Three-quarters of teachers surveyed this year by the New Teacher Project said layoffs should be based on more than just seniority.

Value-added analysis has helped teachers prove their worth and keep their jobs in Tennessee, writes Stephen Sawchuk on Teacher Beat.  The state has collected value-added data for more than a decade, he writes, “and until recently, it was an optional but not mandatory component of teacher evaluations.”

According to (former Tennessee Education Association President Earl) Wiman, over the past decade, the union has actually used information from that state’s value-added system to save teachers’ jobs during tenure and dismissal hearings. In other words, the information showed that those teachers did make a difference for kids, and effectively served as a type of check on principals.

The TEA successfully opposed basing teacher pay on value-added scores — unless that’s negotiated by local unions in districts receiving Race To The Top funds.

On the flip side, Arthur Goldstein describes how a teacher could exploit performance pay to make more money at the expense of students’ learning.

Filed Under: Education Tagged With: , , , , ,

Assessing a teacher’s value

September 8, 2010 By

Value-added analysis of teachers’ work is the subject on Room for Debate.

How should this information be used? What are the strengths and pitfalls of this kind of measurement? If it has flaws, can it be improved and made into a worthwhile tool?

Advocates, like Amy Wilkins of Education Trust, say value-added measures coupled with “rigorous classroom observation” provide valuable feedback for teachers.

When summed over several years, these data can provide teachers with valuable feedback about what kinds of students they are most successful with and with whom they need to improve. They can help schools match the most able teachers with the students who most need them. And they can help leaders better target teacher supports and rewards.

Critics, such as Stanford’s Linda Darling-Hammond, think the method is too unreliable to be useful.

While scores may play a role in teacher evaluation, they need to be viewed in context, along with other evidence of the teacher’s practice.

Better systems exist — like the career ladder evaluations in Denver and Rochester, the Teacher Advancement Program and the rigorous performance assessments used for National Board Certification, all of which link evidence of student learning to what teachers do in teaching curriculum to specific students. These systems also help teachers improve their practice  –  accomplishing what evaluation, ultimately, should be designed to do.

Notice that Wilkins supports value-added scores and classroom observation, while  Darling-Hammond prefers observation but concedes a role for test scores in teacher evaluation. Is a fuzzy consensus emerging?

Filed Under: Education, Testing Tagged With: ,

LA teachers respond to ratings

September 1, 2010 By

Los Angeles teachers rated on effectiveness by the LA Times respond to the value-added evaluations. Some teachers are pleased to be recognized as effective; others feel their hard work and commitment has been disrespected.

Here’s a link to all the teacher responses.

Many teachers do not understand value-added analysis: They think they’re being judged on students’ scores, not on whether students performed as well in their class as they did in previous years.

A few teachers raised real issues: The data may ignore team teaching, the availability of tutoring or the presence of exceptionally disruptive students who make it much harder for their classmates to progress. Teachers with very high-scoring students may not be able to show improvement.

Filed Under: Education Tagged With: , , ,

Evaluating teachers

August 31, 2010 By

While the debate rages about value-added analysis of Los Angeles’ teachers, NPR looks at how value-added data is used in North Carolina’s Winston-Salem/Forsyth County School District. The district began using the data three years ago, notes Robert Siegel, the host. The information is not made public, explains Superintendent Donald Martin

Dr. MARTIN: . . . if you’re red, your students are performing two standard errors below your — sort of comparable counterparts. If you’re yellow, you’re right in the average performance. And if you’re green, you’re two standard errors above.

And if a teacher has one red, you know, their first year, then we literally just have a – it’s like a growth conference with them. They have a personal, you know, individual plan. We talk to them about what are they going to do differently next year.

Then in the second year, if there’s two reds in a row, the teacher has consecutive reds, then we have a trigger for what we call a plan of assistance. And that plan of assistance may involve going to training. It may involve sending in some central office folks to work with that person and to really work on, you know, a very formal plan that’s now, you know – could trigger dismissal at the end of the year if it is unsuccessful.

Principals rarely are surprised by which teachers are red or green, Martin says. But, without data, teacher evaluations suffer from “a Lake Wobegon issue. Everybody is above average.” Administrators are to blame for failing to be honest about teacher effectiveness.

Value-added data is available only for a fraction of teachers, writes Sara Mead on Policy Notebook. She’s concerned about the validity of classroom observations.

There is currently no value-added data for kindergarten and early elementary teachers, teachers in non-core subjects, or high school teachers in most places. My brother-in-law, who teaches middle school band and drama, and sister, who teaches high school composition and literature, do not have value-added data.

When available, value-added data should be used to “inform teacher evaluations,” Mead writes, but the larger issue is developing ways to evaluate all teachers. For example, the Classroom Assessment Scoring System (CLASS) measures the extent to which teachers are teaching in ways linked to improved student outcomes.  Mead is concerned “that the observational rubrics many districts and states will put into place under their proposed evaluation systems have not yet been validated.”

While an Economic Policy Institute report urges caution in relying on value-added data, others say the alternative ways to assess teachers, such as classroom observations, are much less reliable than value-added, notes Teacher Beat.  “I think people are right to point out the potential flaws of [value-added modeling], but it should be compared against what we have, not some nirvana that doesn’t exist,” said Daniel Goldhaber, a professor at the University of Washington in Bothell.

In response to teacher feedback, Houston Superintendent Terry Grier has told principals to collaborate with teachers on an individual plan setting out each teacher’s goals for the year and how the principal will help the teacher meet them.  The Houston Federation of Teachers sees this as a nefarious plot to make teachers look bad, writes Rick Hess. HFT is telling teachers not to admit to any performance weaknesses or allow test scores to be used to judge their success.  There’s a lot of fear out there.

Update: Here’s the New York Times’ value-added story.

Filed Under: Education Tagged With: , , , , ,

No gold stars for LA teachers

August 30, 2010 By

Los Angeles doesn’t reward, recognize or try to learn from its most effective teachers, reports the LA Times in a follow-up to its value-added analysis of third- through fifth-grade teachers’ effects on their students’ test scores.

The Times found that the 100 most effective teachers were scattered across the city, from Pacoima to Gardena, Woodland Hills to Bell. They varied widely in race, age, years of experience and education level. They taught students who were wealthy and poor, gifted and struggling.

In visits to several of their classrooms, reporters found their teaching styles and personalities to differ significantly. They were quiet and animated, smiling and stern. Some stuck to the basics, while others veered far from the district’s often-rigid curriculum. Those interviewed said repeatedly that being effective at raising students’ performance does not mean simply “teaching to the test,” as critics of value-added analysis say they fear.

On average, these teachers’ students improved by 12 percentile points on tests of English, from the 58th to the 70th, and 17 percentile points in math, from 58th to 75th, in a year.

Thomas Kane, a Harvard education researcher, tested the reliability of the value-added approach in Los Angeles, the Times reports.  Kane predicted the student gains for  156 teachers who volunteered for the experiment.

Value-added analysis was a strong predictor of how much a teacher would help students improve on standardized tests. The approach also controlled well for differences among students, the study found.

With $45 million from the Bill and Melinda Gates Foundation, Kane and other researchers are now following 3,000 teachers in six school districts to see if other types of evaluation — including sophisticated classroom observations, surveys of teachers and reviews of student work — are also good measures of teacher performance.

In the meantime, Kane said that, although it is not perfect, “there is currently not a better measure of teacher effectiveness than the value-added approach.”

Teachers on the Times’ most effective list said they’d never been recognized for excellence.  Aldo Pinto, a 32-year-old teacher at Gridley Street Elementary School in San Fernando, said, ”The culture of the union is: Everyone is the same. You can’t single out anyone for doing badly. So as a result, we don’t point out the good either.”

Value-added is the worst form of teacher evaluation, but it’s better than everything else, writes Chad Aldeman on The Quick and the Ed.

Los Angeles Unified now plans to share value-added data with teachers privately and hopes to negotiate its use in teacher evaluations with the teachers’ union.  Tennessee did just the opposite, Aldeman notes. “Every year since the  mid-1990’s every single eligible teacher has received a report on their (value-added) results.”

When these results were first introduced, teachers were explicitly told their results would never be published in newspapers and that the data may be used in evaluations. In reality, they had never really been used in evaluations until the state passed a law last January requiring the data to make up 35 percent of a teacher’s evaluation. This bill, and 100% teacher support for the state’s Race to the Top application that included it, was a key reason the state won a $500 million grant in the first round.

While LA teachers are angry and confused, Tennessee teachers have had time to understand how value-added analysis works and  prepare to accept it.

Filed Under: Education Tagged With: , , , , ,

LA Times lists ‘effective’ teachers, schools

August 29, 2010 By

The Los Angeles has posted its list of the most and least effective third-, fourth- and fifth-grade teachers and schools with teachers’ comments. A few teachers are challenging the data, saying that they’re listed as teaching in years when they were on leave. The value-added analysis of schools is interesting.

On City Journal, dducation researcher Marcus Winters looks at the pros and cons of value-added analysis and comes out against release individual teachers’ scores. 

 Test-score analysis is “correct” on average—it can tell us a great deal about aggregate teacher quality. It can also help to evaluate individual teachers. But given its messiness—especially when tied to stakes as high as people’s jobs—it cannot be used in isolation.

Critics go too far, however, when they claim that these limitations justify abandoning the value-added approach altogether. The real lesson is that test scores are best used to raise red flags about a teacher’s objective performance; rigorous subjective assessment should follow, to ensure that the teacher is truly performing poorly. If both analyses show that a teacher is ineffective, then action should be taken, including removal from the classroom.

Economic Policy Institute also sees Problems with the Use of Student Test Scores to Evaluate Teachers.

Filed Under: Education, Testing Tagged With: , , ,

AFT: Don’t publish teachers’ names

August 19, 2010 By

Parents have a right to know how their children’s teachers are rated on employee evaulations, Randi Weingarten, head of the American Federation of Teachers, told the Los Angeles Times. However, she asked the Los Angeles Times not to go forward with plans to publish the names of more than 6,000 elementary school teachers ranked by their record of improving students’ reading and math scores over a seven-year period.

Teachers “look at this as a hammer, a sledgehammer, and they’re scared about it,” she said. “They’re schoolteachers; they’re private individuals…. They’re not public figures.”

More than 1,100 teachers have requested and received copies of their value-added rankings, the Times reports. “More than 100 have submitted comments on their rankings that will be published as part of the database.”

When done properly, value-added analysis could be a valuable part of assessing teacher performance, Weingarten said.

“There’s a right way to do evaluation, and we have to keep everybody’s feet to the fire,” she said.

She added that the system of teacher evaluations had been “broken for years,” and needed drastic reform. She said a good system of teacher evaluations would ensure that struggling teachers receive the help they need to improve, but would also make it easier to fire teachers who were unable to change.

Third Street Elementary teacher Karen Caruso, who was named in the Times’ story as ranking among the bottom 10% of elementary teachers in the value-added analysis, is   “the most beloved teacher in the school,” Weingarten said.

(Caruso) is known for helping her students become more critical thinkers and better problem solvers — skills, she implied, that wouldn’t necessarily be reflected in standardized test scores.

United Teachers of Los Angeles strongly opposes efforts to consider student test scores in teacher evaluations. The union has called on teachers to boycott the Times.

On California Watch, Louis Freedberg looks at the ethics of “outing teachers” and questions the validity of value-added analysis; a Times reporter responds.

Filed Under: Education, Testing Tagged With: , , ,

Duncan backs release of teacher data

August 17, 2010 By

In response to the Los Angeles Times story naming effective and ineffective teachers (based on value-added analysis), Education Secretary Arne Duncan said parents have a right to know how well their teachers are doing at raising students’ test scores.

“What’s there to hide?” Duncan said in an interview. “In education, we’ve been scared to talk about success.” And failure.

After years of ignoring the data, LA will be forced to do something about chronically ineffective teachers, writes Jay P. Greene.

No one is suggesting that analyses of these test scores should be the sole criteria for identifying or removing ineffective teachers.  But it is a start.

This is going to spread.  As long as the data exist, there will be more and more pressure for school systems to actually use the information and develop systems for identifying and removing teachers who can’t teach.

Flypaper questions naming teachers, but adds that the story “makes teachers, even the struggling ones, look better and the district and union look worse.”

The district and union have acted shamefully by not only hiding performance data from the public but hiding them from individual teachers themselves — teachers who, it appears, want to see them and want to learn from them.

Teachers should be protected from public reprimands, writes John Thompson on This Week in Education. Duncan’s endorsement of the story lacked “common decency,” he adds.

I’ve added more reaction to the Times’ story to my original post.

United Teachers of Los Angeles, the union, is “really, REALLY peeved” by the “potentially explosive” story, writes Stephen Sawchuk at Teacher Beat. There’s talk the union will ask teachers to boycott the LA Times. But now that people know the data is available, they’re going to want to use it, he predicts.

Rick Hess thinks value-added analysis isn’t completely ready for prime time and says the Times should not have used teachers’ names. I have to say naming the teachers made me uncomfortable.

Dan Willingham also thinks value-added analysis is not good enough to use for evaluating individual teachers, though his example doesn’t make sense to me. He adds that a consensus has emerged that something has to be done about incompetent teachers. Most districts don’t have “a mechanism by which to ensure that incompetent teachers are not teaching.”

I have said before that if teachers didn’t take on the job of evaluating teachers themselves, someone else would do the job for them.

. . . This is the time for the teacher’s unions to make teacher evaluation their top priority. If they don’t, others will.

I don’t believe the teachers’ unions can take on this challenge, though they’d be wise to try.

Filed Under: Education Tagged With: , , , ,