Good teaching, poor test scores

Evaluating teachers based partly on student test scores is unreliable, concludes a study in Educational Evaluation and Policy Analysis. Researchers analyzed a subsample of 327 fourth- and eighth-grade mathematics and English-language-arts teachers across six school districts.

“Some teachers who were well-regarded based on student surveys, classroom observances by principals and other indicators of quality had students who scored poorly on tests,” reports the Washington Post. Some poorly regarded teachers had students who did well.

Thirty-five states and the District of Columbia require student achievement to be a “significant” or the “most significant” factor in teacher evaluations. Just 10 states do not require student test scores to be used in teacher evaluations.

Most states are using “value-added models” — or VAMs — which are statistical algorithms designed to figure out how much teachers contribute to their students’ learning, holding constant factors such as demographics.

Last month, the American Statistical Association warned against used VAMS, saying that “recent studies have found that teachers account for a maximum of about 14 percent of a student’s test score.”

“We need to slow down or ease off completely for the stakes for teachers, at least in the first few years, so we can get a sense of what do these things measure, what does it mean,” said Morgan S. Polikoff, a USC assistant professor of education and co-author of the study. “We’re moving these systems forward way ahead of the science in terms of the quality of the measures.”

Schools get D+ from Students First

The nation’s schools earn a D+  from Michelle Rhee’s Students First. No state earned an A, reports U.S. News.

The group evaluated states on three policy areas: how well states “attract, retain and recognize quality teachers,” how well they give parents easily accessible information about their children’s schools and how well they spend public funds to support schools and teachers.

Louisiana (B-) and Florida (B-) earned the highest grades, followed by Indiana (C+). North Dakota, Montana and Vermont received F’s.

Fourteen states now assign A-F letter grades to schools or will do so by 2015, reports the Education Commission of the States’ new accountability database.

? All 50 states and the District of Columbia consider student achievement as measured by test results in their performance indicators
? 37 states and D.C. factor in student growth or improvement on tests in deciding school performance. That’s up from 21 in 2002.
? 44 states and D.C. consider graduation rates in determining school performance while 12 states include dropout rates.
? 9 states weigh growth of the lowest-performing quartile of students in judging their schools.

Florida was the first state to issue letter grades to schools in 2002.

Schools raise scores, but not smarts

Schools can improve students’ achievement test scores, but not their cognitive ability, writes Scott Barry Kaufman in Scientific American. Reseachers analyzed math and English scores and cognitive ability (working memory, processing speed, and abstract reasoning) among nearly 1,400 eighth graders attending traditional, exam and charter public schools in Boston.

“Good test takers tend to have high levels of working memory, processing speed, and abstract reasoning skills,” Kaufman writes.

Cognitive ability was associated with growth in achievement test scores from 4th to 8th grade. This is consistent with prior research suggesting that cognitive ability predicts academic achievement, but academic achievement does not predict cognitive ability.

Students in some schools showed growth in achievement scores but school quality “played little role in the growth of cognitive ability.”

Students attending a charter school as a result of winning the admissions lottery had higher standardized test scores compared to students who lost the lottery.

There was no difference between the lottery groups, however, on measures of cognitive ability.

Cognitive skills such as fluid reasoning and executive functioning (working memory and cognitive inhibition) affect many life outcomes, from school performance to drug use, Kaufman writes. The researchers cite “examples of targeted programs that increase cognitive control and reasoning.”

Christmas cheer raises scores

Christmas cheer raises test scores, concludes Brookings’ Matthew Chingos.

He crunches PISA data to show that scores are higher in countries where Christmas is a public holiday. (First step: Exclude Shanghai.)

That’s confirmed by NAEP scores on fourth-grade math performance from 1990 to 2013, which show test scores rise and fall with holiday cheer (measured by consumer spending in November and December).

Standardizing the NAEP scores and putting the spending index on a logarithmic scale implies that if we could just have about 30% more holiday spirit, our students would do as well as those in Finland!

Brilliant, writes Jay Greene. And the reason why “random-assignment and other research designs that more strongly identify causation are so important.”

How parents choose schools

Georgia parents don’t choose private schools for their test scores, concludes More Than Scores, a study of the the state’s tax-credit scholarship program by the Friedman Foundation for Educational Choice.

Parents who chose to use the scholarships at private schools cared most about disciplinary policies, learning climate, class sizes, safety and individual attention for their children.

Since 2008, Georgia students have been able to receive scholarships to private schools through nonprofits, which are funded by individual and corporate contributions. Donors get an offsetting state income tax credit.

Only 10.2 percent rated “higher standardized test scores” as one of their top five reasons for choosing a private school. Parents were most concerned about finding a safe, orderly school.

Most popular among respondents were:

“better student discipline” (50.9 percent),

“better learning environment” (50.8 percent),

“smaller class sizes” (48.9 percent),

“improved student safety” (46.8 percent), and

“more individual attention for my child” (39.3 percent).

Low-income parents give top priority to graduation rates and college acceptance rates in deciding on a school.

Elite colleges ask more of homeschoolers

Are Elite Colleges and Universities Discriminating Against Homeschoolers? asks Paula Bolyard, a recently “retired” homeschooler, on PJ Lifestyle.
Cover edit 3

Homeschooled student “enter college with significantly higher test scores than their public (and even private) school peers,” she writes. “They graduate from college at a higher rate­—66.7 percent compared to 57.5 percent—and earn higher grade point averages while in school, according to one study.”

Princeton seems to get it, she writes. Applicants who can’t supply a traditional transcript can submit an outline of the homeschool curriculum.

Yale wants to make sure homeschooled kids are not socially awkward:

We look for evidence of social maturity from all our applicants and especially from home-schooled students. Your personal statement, interests and activities, and letters of recommendation should speak to your ability to integrate well with other students and tell us about your non-academic interests.

But elsewhere Yale says “academic strength” is the “first consideration” with “motivation, curiosity, energy, leadership ability, and distinctive talents” in second place.

“We’re going to want to know what the reason for homeschooling is,” a Dartmouth admissions official told Lindsay Cross at the Mommyish blog.

“Was the student busy with another demanding pursuit, like playing music? Were they traveling with their family? Was there a lack of resources in their area? Somewhere in the application, they’re going to need to explain.”

Private school students aren’t asked to explain why they didn’t attend public school, Bolyard points out.

Some elite colleges ask homeschooled students to submit additional SAT II test scores. That strikes me as reasonable. A straight-A student who’s been graded by Mom will need objective evidence of achievement.

But what about a teacher’s recommendation when Mom is the teacher?

In addition to a “not-so-subtle interrogation about the family’s choice to opt out of public education,” Brown also asks for “letters of recommendation from instructors who have taught you in a traditional classroom setting and who can speak to your abilities and potential in an objective way.”

Brown “would prefer not to receive letters of recommendation from your parents, immediate relatives, or from academic tutors in the paid employ of your family,” unless the applicant has no classroom instructors to ask.

Testing plus school inspections?

The English use school inspectors as well as centralized testing to provide feedback to principals, inform parents and identify “serious weakness,” writes Iftikhar Hussain in Education Next.

Inspectors visit schools once in three to six years. They observe classes, interview school leaders, examine students’ work and talk to students and parents. In addition, students take a national test at age 7, 11, 14, and 16.

From 2006 to 2009, 13 percent of schools were rated Outstanding, 48 percent Good and 33 percent were Satisfactory. Six percent failed: 4.5 percent were moderate fails and 1.5 percent required “special measures.”

Schools that receive a moderate fail rating are subject to additional inspections, with an implicit threat of a downgrade to the severe fail category if inspectors judge improvements to be inadequate. Schools that receive the severe fail rating may experience more dramatic consequences: these can include changes in the school leadership team and the school’s governing board, increased resources, as well as increased oversight from the inspectors.

Inspector ratings are correlated with student- and parent-reported measures of school quality, even after controlling for test-score results and other school characteristics, writes Hussain.

Fail ratings lead to test score improvements that are especially large for the lowest-scoring students, Hussain finds.  “These results are consistent with the view that children of low-income parents, arguably the least vocal in holding teachers accountable, benefit the most from inspections.”

Some U.S. education reformers see school inspectors as a way to hold schools accountable without relying exclusively on test scores.

If parents fail, schools will too

In Raising the Curve: A Year Inside One of America’s 45,000 Failing Public Schools, Ron Berler takes readers inside Brookside Elementary in Norwalk, Connecticut, which is trying to raise chronically low test scores. About half the students come from Hispanic immigrant families; 41 percent qualify for a subsidized lunch.

The book follows two fifth-grade friends: Hydea does her work, but has only second-grade reading skills. Marbella is only a year behind in reading, but would rather obsess about her social life — and Justin Bieber — than do homework.

The literacy teacher works with teachers to improve their skills, but breaks away before the state exams to coach a small group of children who are close to passing. She has no time for those who are way behind or for children in untested grades. Some students have been passed along with subpar skills. Others have received tutoring or summer school help, but remain behind.

Berler, who volunteered as a mentor and teacher’s aide at the school, is sympathetic to teachers and the principal, but frustrated with uninvolved parents. Some don’t know how to help, he writes. Others are too busy working to supervise their children closely.

When fifth-grade teacher Keith Morey asks students about their responsibilities at home, a clear pattern emerges. The responsible students are expected to do chores; the kids who don’t do homework don’t work at home either.

Schools need to engage parents and teach them how to support their children’s learning, writes Berler in the Huffington Post.

Gates: Mix measures to evaluate teachers

Combining growth in students’ test scores, student feedback and classroom observations produces accurate information on teacher effectiveness, according to Gates Foundation research.

A composite measure on teacher effectiveness drawing on all three of those measures, and tested through a random-assignment experiment, predicted fairly accurately how much high-performing teachers would successfully boost their students’ standardized-test scores, concludes the series of new papers, part of the massive Measures of Effective Teaching study launched three years ago.

No more than half of a teacher’s evaluation should be on growth in student achievement, researchers concluded.  In addition, teachers’ classroom performance should be observed by more than one person.

Of course, the controversy on how to evaluate teachers — and what to do with the information — is not over.

The ever-increasing federal role in education makes no sense, writes Marc Tucker, who complains that U.S. Education Secretary Arne Duncan is forcing states to evaluate teachers based on student performance in order to get No Child Left Behind waivers.  Most researchers don’t think value-added measures of teacher performance are reliable, writes Tucker.

The study is a “political document and not a research document,” Jay Greene tells the Wall Street Journal.  Classroom observations aren’t a strong predictor of student performance says Greene, a professor of education policy at the University of Arkansas. “But the Gates Foundation knows that teachers and others are resistant to a system that is based too heavily on student test scores, so they combined them with other measures to find something that was more agreeable to them,” he said.

Ex-superintendent gets 3.5 years for test scheme

Lorenzo Garcia, former superintendent in El Paso, has been sentenced to sentenced to 3.5 years in prison for fraud for manipulating test scores to collect $56,000 in bonuses and misleading the school board to steer a $450,000 no-bid contract to  his mistress.

Garcia, who was hired in 2006, worked with administrators to pre-test 10th graders and assign low scorers to ninth or 11th grade, so their scores wouldn’t be counted in the critical 10th-grade year. After repeating ninth grade, students were moved to 11th grade. Older students were told to drop out and get a GED.

In some cases, when the district needed to improve its graduation rate, it gave students credit for computer-based classes or “turbo-mesters,” which were 90-minute sessions in which students earned a full semester worth of credits.

“One girl got two semesters in three hours, in the last day of school, while her teacher was collecting books,” said former principal Stephen Lane, one of five people allowed to testify before Briones sentenced Garcia.

When Lane protested Garcia’s methods, he was fired and escorted out of the building by police.

High school test scores rose and El Paso’s rating improved from “academically acceptable” in 2005 to “recognized” in 2010. Then someone noticed all those missing 10th graders.