When to trust (or not) the experts

Dan Willingham’s new book, When Can You Trust the Experts: How to Tell Good Science from Bad in Education is out. (Download chapter 1 here.)

Every new program claims to be “research-based,” writes Willingham. Teachers and administrators don’t have the time to evaluate everything and there are no credible summaries to help.

 The first half of the book focuses on what cues we use to tell us “this is probably true,” and how they can be misleading.

. . . (It also describes) how when can know when science might help with a particular problem and when it can’t.

The second half of the book gives Willingham’s four-step short cut for evaluating research claims: Strip it (of verbiage), trace it, analyze it and should I do it?


$1.1 million to test ‘galvanic’ bracelets

The Gates Foundation is spending $1.1 million to test “galvanic skin response” bracelets that measure students’ engagement in lessons, writes Valerie Strauss on Answer Sheet. Clemson and the National Center on Time and Learning will research the idea’s feasibility.

Strauss sees it as a “nutty” waste of money that could be spent on books, teachers and librarians.

Is it foolish? Let’s say research shows that students learn more in the X state than when their bracelets record Z’s. Teachers could analyze the high-X and high-Z portions of their lessons to figure out how to reach students more effectively. Of course, the idea could be a dud. Maybe too many students X up or Z out for reasons that have nothing to do with learning. But we don’t know that yet.

Using science to improve the art of teaching

Is Teaching an Art or a Science? Cognitive scientist Daniel Willingham discusses how science can be used to improve the art of teaching.

Teacher assigns ‘oppo’ research for Obama

Eighth graders at a Virginia public school were told to research the weaknesses of Republican presidential candidates, write a paper on how to exploit the weaknesses and identify who to send the paper to in the Obama campaign.

“This assignment was just creepy beyond belief — like something out of East Germany during the Cold War,” one frustrated father, who asked for his family to remain anonymous, told The Daily Caller.

Michael Denman divided his honors civics class into four groups, all assigned to do “oppo” research on Republicans. After parents complained, the principal told the teacher he should have let students research a candidate from either party, a Fairfax County Public Schools spokesman said.


Allegedly, the teacher never told students to send their research to the Obama campaign, but why assign two students in each group to figure out the name of the right “oppo” person?


Feds delay, twist Head Start research

Head Start’s benefits fade quickly, writes Jay Greene, but Health and Human Services is “up to its old tricks of delaying research whose results are likely to undermine their darling program, Head Start.”

In a letter to HHS Secretary Kathleen Sebelius, five senators asked “why the latest round of results of the congressionally mandated study have not been released four years after data collection was complete and one year after the report was scheduled to be released.”

In 2010 I told you about how the Department of Health and Human Services delayed the release of the previous round of disappointing research results about the lasting effects of Head Start.  When the extremely high quality study, involving a random-assignment design on a representative sample of all Head Start programs nationwide, was finally released three years after the data collection was complete, it found that students randomly assigned to Head Start performed no better on cognitive measures by the end of kindergarten and first grade.

Despite this, HHS declared the program a huge success.  “Research clearly shows that Head Start positively impacts the school readiness of low-income children,” said Sebelius.


Review: 75% of charter studies are flawed

Seventy-five percent of charter school studies are flawed because they fail to account for charter students’ differences in academic background and performance, according to a meta-analysis published in Science.

High-quality research is emerging from charters that use lotteries to pick students, write Julian R. Betts, a professor of economics at the University of California, San Diego, and Richard C. Atkinson, a former president of the University of California system who once served as director of the National Science Foundation. Students who apply to a charter but lose the lottery represent a sound control group, they write.

The relatively small number of lottery-based studies of charter schools have generally shown that they either outperform or perform at the same level as traditional public schools, according to the authors. But those studies cover only a small fraction—about 2 percent—of charter schools nationally.

However, charters that need lotteries for admission may be unusually good schools, the authors warn.

NEPC: Base productivity ideas on research

The U.S. Department of Education’s Increasing Educational Productivity project, which provides dollar-stretching advice to school districts, isn’t backed by solid research, charges a National Education Policy Center report.

Researchers have ignored efficiency and productivity “over the last half-century,” responds Rick Hess. “Most cost-saving efforts in most sectors are based on sensible intuitions and experimentation rather than “rigorous science.”

Education reform’s future

It’s not quite the lion lying down the lamb, but Rick Hess of the American Enterprise Institute and Linda Darling-Hammond, a Stanford ed professor who served on Obama’s transition team, have co-written a New York Times op-ed, How to Rescue Education Reform.  They disagree on some key issues, but agree that the federal government should stick to what it alone can do and avoid trying to micromanage schools.

The first federal role is transparency:  No Child Left Behind required states to measure and report achievement, so parents, voters and taxpayers could “hold schools and public officials accountable.” However, states were allowed to set their own, low standards.

Instead of the vague mandate of “adequate yearly progress,” federal financing should be conditioned on truth in advertising — on reliably describing achievement (or lack thereof) and spending. To track achievement, states should be required to link their assessments to the National Assessment of Educational Progress (or to adopt a similar multistate assessment). To shed light on equity and cost-effectiveness, states should be required to report school- and district-level spending; the resources students receive should be disclosed, not only their achievement.

The second federal role is “enforcing civil rights laws and ensuring that dollars intended for low-income students and students with disabilities are spent accordingly.”

Third is supporting basic research in fields such as “brain science, language acquisition or the impact of computer-assisted tutoring.”

Competitive federal grants can support innovation, they conclude. However, the “Obama administration’s $4.35 billion Race to the Top competition . . .  ended up demanding that winning states hire consultants to comply with a 19-point federal agenda, rather than truly innovate.”

The feds should stop trying to improve schools by order from above, write Hess and Darling-Hammond. “The federal government can make states, localities and schools do things — but not necessarily do them well.”

Schizophrenic, responds RiShawn Biddle.

The odd couple call adequate yearly progress a “vague mandate,” but elsewhere  complain it’s too prescriptive, writes Andrew Rotherham.  The left and right are uniting to kill education reform, he adds in Time.


What works for teachers? Let’s find out

“Teacher moves” — the many decisions a teacher makes every day — have been ignored by education researchers, writes Michael Goldstein, who founded the high-scoring MATCH charter school in Boston, in Education Next.

Should I ask for raised hands, or cold-call? Should I give a warning or a detention? Do I require this student to attend my afterschool help session, or make it optional? Should I spend 10 minutes grading each five-paragraph essay, 20 minutes, or just not pay attention to time and work on each until it “feels” done?

Researchers care about raising test scores, while teachers “care more about solving today’s problems,” Goldstein writes.

Teachers need to use time efficiently. Researchers don’t consider opportunity cost: They want teachers to spend more time on X without saying where they should spend less time.

Education researchers often put forward strategies that make teachers’ lives harder, not easier. Have you ever tried to “differentiate instruction”? When policy experts give a lecture or speak publicly, do they create five different iterations for their varied audience? Probably not.

 Experienced teachers have seen fads, allegedly supported by research, come and go. Newbies pick up the veterans’ skepticism. To develop useful research, “it makes sense to focus on topics that teachers care about,” writes Goldstein.

1. How to be more efficient. Many teachers want to work less without being neglectful. Or they’d like to free up time to invest in new priorities.

2. How to manage the classroom so kids behave better, thus lowering the “misbehavior tax” on learning. If a middle school teacher can “reset” the class only 3 times per period, instead of 5, that’s probably 1,440 fewer times per year that he has to deal with misbehavior. (By “reset,” I mean when a teacher says something like, “Guys, come on. I need your eyes on me. I need you to settle down. Joey, that means you. I’m going to wait until I have everyone’s eyes.”)

3. How to motivate and generate student effort, especially, how to “flip” kids who arrive having not worked hard in previous classes or years. This includes both getting kids to exert effort during class and getting them to work hard at home.

4. How to get kids to remember material that they seemingly once knew. Cognitive science has moved the ball forward here; now we need applied experiments with teachers.

5. How to best explain particular ideas and concepts. Each year, tens of thousands of math teachers try to get kids to understand the notion that division by zero does not exist.

Goldstein proposes the Teaching Move Genome Project to identify teacher moves and rate their effectiveness. He wants randomized trials, not just perceived best practices.

In his best-selling book Teach Like a Champion, Doug Lemov describes 49 teaching moves he has observed in the nation’s top charter schools. At the University of Michigan, Deborah Ball and her colleagues are close to unveiling a list of 88 math teacher moves. Lee Canter’s Assertive Discipline and Jon Saphier’s Skillful Teacher discuss scores of moves, like the “10-2” rule (have kids summarize for 2 minutes in small groups after 10 minutes of teacher-led instruction), much of it supported by nonrandomized research.

Goldstein envisions a randomized trial of Lemov’s  “Right Is Right” move.

The idea is that when a kid gives an answer that is mostly right, the teacher should hold out until it’s 100 percent correct. Lemov describes various tactics the teacher can use to elicit the 100 percent right answer from the student (or first from another student, before having the original student repeat or extend the correct answer).

The obvious cost of implementing this move is time. These back-and-forths add up to lost minutes each period when other topics are not being discussed. A less skillful teacher might be drawn into a protracted discussion, when her next best alternative (simply announce the 100 percent right answer, and move on) might work better. We just don’t know.

Is it feasible to test the effectiveness of “teacher moves” for different teachers and different sets of students?  Is it a good use of time?

Researchers: Single-sex ed is ‘pseudoscience’

Single-sex education is based on “pseudoscience,” charge a team of neuroscience and child development experts in a Science article. There is “no empirical evidence” that segregating students by sex improves education, they argue. There’s plenty of  evidence it can increase gender stereotyping among students and adults.

The National Association for Single-Sex Public Education estimates more than 500 schools separate boys and girls for at least some classes, reports Inside School Research.

A new curriculum may yield a short-term gain because it’s evaluated by true believers, the scientists said.

“Novelty-based enthusiasm, sample bias, and anecdotes account for much of the glowing characterization of [single-sex] education in the media. Without blind assessment, randomized assignment to treatment or control experiences, and consideration of selection factors, judging the effectiveness of innovations is impossible.”

“There are some definite brain differences in boys and girls as children, but there are a lot of overlaps, and there’s absolutely nothing to suggest that they learn differently,” Claremont McKenna Psychology Professor Diane Halpern told Inside School Research. “The underlying biology of learning is the same.”

Students in single-sex classes don’t perform significantly better than those in mixed-gender classes, once the students’ prior performance and characteristics are taken into account, the critics said.

Update: If there’s no evidence single-sex education is any worse than mixed classes — and there isn’t — then let parents decide, responds Paul Peterson on Ed Next. Many parents like the idea for a variety of reasons, he writes.