Teachers can learn from tests

Once a foe of standardized testing, Ama Nyamekye improved her teaching by analyzing her students’ scores on New York’s Regents exam, she writes in Ed Week.  When she asked her sophomores to take the English Regents exam a year early, she discovered “holes in my curriculum.”

I once dismissed standardized testing for its narrow focus on a discrete set of skills, but I learned that my self-made assignments were more problematic. It turned out they were skewed in my favor. I was better at teaching literary analysis than grammar and punctuation. When I started giving ongoing standardized assessments, I noticed that my students showed steady growth in literary analysis, but less growth in grammar and punctuation. I was teaching to my strengths instead of strengthening my weaknesses.

Grading is subjective, she writes. Emotionally invested in her students’ success — and implicitly judging her own effectiveness — she was quick to see signs of achievement.

By contrast, her students’ Regents essays were graded by English teachers who didn’t know them and who used detailed rubrics.

When I “depoliticized” the test, I found a useful and flawed ally. The exam excelled where I struggled, offering comprehensive and standards-based assessments. I thrived where the test fell short, designing creative, performance-based projects. Together, we were strategic partners. I designed and graded innovative projects—my students participated in court trials for Shakespearean characters—and the test provided a rubric that guided my evaluation of student learning.

All her students who took the exam passed it. Most earned high scores.

‘Crazy U’ for college-crazed parents

Andrew Ferguson’s Crazy U: One Dad’s Crash Course in Getting His Kid Into College is getting great reviews.

The New York Times compares his writing to Mark Twain, Tom Wolfe and Dave Barry.

The admissions process, as Andrew Ferguson puts it in his new book, “Crazy U,” entangles not just our pocketbooks but everything else that, besides world peace and cocktail hour, matters to parents: “our vanities, our social ambitions and class insecurities, and most profoundly our love and hopes for our children.”

. . . As this story moves forward, Mr. Ferguson makes short, shrewd detours into areas that include: the history of American education, how college guidebooks compile their rankings, the SAT tests and its critics, and the headache-making intricacies of college loans and financial aid. He talks to an expensive admissions guru who learns of his late start and fumbling progress and says, smiling: “Oooooh. Baaaaaaad Daaaaaad.”

The book is “compulsively readable, unusually vivid — and thoroughly dispiriting,” concludes the Wall Street Journal.

This is a guy who doesn’t just delve into the history of the SAT. He also takes the test himself. (“Close to a disaster,” he says of the results, with a math score so bad that he won’t divulge it, other than to say “somewhere below ‘lobotomy patient’ but above ‘Phillies fan.’ “)

. . . A series of enervating campus visits is marked by interchangeably chirpy undergraduate tour guides united by their ability to walk backward while extolling the school’s a capella groups and reassuring parents about the high priority placed on security. On a swing through New England, the Fergusons narrowly miss Dartmouth’s Second Annual Campus Sex Screening, a supposedly health-promoting event where, the flyers promised, “sexperts” would be giving “free demonstrations!” and the party favors included dental dams, glow-in-the-dark condoms and Ben & Jerry’s ice cream. Mr. Ferguson muses: “I may be showing my age, but back when I was a college student we didn’t need free ice cream to get us to come to a sex demonstration.”

The Washington Post reviewer, whose daughter is waiting to hear from colleges,  is rooting for Dad.

There’s the son telling his high school counselor that in college he wants to major in beer and paint his chest in the school colors at football games, prompting Dad to snap later: “It’ll be a big help when he writes your recommendation.”

Then there’s Dad handing his procrastinator a book on successful college essays and watching the boy vacantly turn it over in his hands. “I thought of the apes coming upon the obelisk in the opening scene of ’2001: A Space Odyssey,’ ” Dad writes. “He did everything but sniff it.” And here’s Dad encountering a mother who gloats that she and her daughter worked three solid months on the essays every day after school, plus weekends. “We did three months of work too,” he tells her, “in twelve days.”

My review:  This really is a great read for college-crazed parents and those about to enter the fray. It’s all 12 years behind me now, but I remember the craziness.

DePaul: Show ‘heart,’ not SATs

SAT or ACT scores will be optional for DePaul University applicants starting next year, but those who choose not to submit scores will be asked to write short essays demonstrating “noncognitive” traits such as leadership, commitment to service and ability to meet long-term goals. From the Chronicle of Higher Education:

“Admissions officers have often said that you can’t measure heart,” said Jon Boeckenstedt, associate vice president for enrollment management. “This, in some sense, is an attempt to measure that heart.”

DePaul hopes to encourage more applications from low-income and minority students with relatively high grades and low test scores.  “Heart” is a better predictor of success than SAT or ACT scores for low-income and minority students, admissions officials say. 

In 2008 the university added four short essay questions to its freshman application with hopes of assessing noncognitive traits said to lead to college success.

One question prompted applicants to describe a goal they had set for themselves and how they planned to accomplish it: “How would you compare your educational interests and goals with other students in your high school?” Another question said: “Describe a personal challenge you have faced, or a situation in which you or others were treated unfairly. How did you react to the situation and what conclusions did you draw from the experience? Were you able to turn to others for support?”

DePaul dropped those questions when it started using the Common Application, which requires a personal essay of at least 250 words. It was too much writing.  The questions will return for students who don’t submit test scores.

Race to new tests

Competition has opened for $350 million in Race To The Top funding for new assessments linked to common standards, reports Education Week. That means less multiple-choice testing  and more “essays, multidisciplinary projects, and other more nuanced measures of achievement.”

(The Education Department) wants tests that show not only what students have learned, but also how that achievement has grown over time and whether they are on track to do well in college. And all that, the regulations say, requires assessments that elicit “complex student demonstrations or applications” of what they’ve learned.

There is money for “comprehensive assessment systems” measuring mastery of a “common set of college- and career-ready” standards. Applicants get points for working with state universities to design the tests and guarantee that students who score above a certain level will be able to enroll in for-credit college classes.

Another pot of money will fund end-of-course high school exams.

Stanford Education Professor Linda Darling-Hammond, who leads a group representing a majority of states, believes performance assessments can improve the way teachers teach, notes John Fensterwald on Educated Guess.

The alternative is performance assessments, which require students to construct their own responses to questions. These can take the form of supplying short phrases or sentences to questions, writing essays or conducting complex and time-consuming activities, such as a lab experiment. “By tapping into students’ advanced thinking skills and abilities to explain their thinking, performance assessments yield a more complete picture of students’ strengths and weaknesses,” Darling-Hammond wrote.

“Performance assessments face obstacles of cost, reliability and testing time,” Fensterwald writes. He links to a critique of Darling-Hammond’s paper by Doug McRae, a retired publisher for the testing division of McGraw-Hill.

Because multiple-choice questions are cheap and easy to score, it’s possible to ask students a wide range of questions. As tests get more complex — write an essay, design an experiment, stage a debate — students  spend more time being assessed on far fewer prompts. Grading is subjective. Todd Farley’s Making the Grades explains tough it is for a group of people to score short answers and essays with consistency and fairness.

Getting in without SATs

Sarah Lawrence, a small liberal-arts college, picks admits without considering SAT scores. With grades varying so much from school to school, the admissions committee uses “a sample essay graded by a high-school teacher to determine the curriculum’s rigor,” New York Magazine explains.

But the samples also tell something about the readers. “I had one essay that said how awful Twilight was”—the essay was about damaging themes of female submissiveness in the series—“and I was like, ‘Admit her!’?” says Melissa Faulner, a 2006 grad on the committee. Whereas what the readers wryly call TCML essays—“theater changed my life”—are looked at more skeptically.

A girl from Texas scored a three (out of five) in academics while getting top marks in the other two categories. “Her grades really are bad,” Will Floyd allowed. She hadn’t gotten one A in high school. “But her writing was gorgeous,” he noted. The girl explained in her application that she has test anxiety and problems with rote memorization. But she had good recommendation letters. Besides, Sarah Lawrence’s curriculum emphasizes writing over test-taking. She got in.

More than half of applicants are offered a place at Sarah Lawrence.  Tuition and room and board cost more than $55,000 a year: 61 percent of undergrads receive financial aid.

Contest time

The Profile in Courage essay contest, sponsored by the John F. Kennedy Presidential Library & Museum, invites high school students to write an original essay about an elected official who has demonstrated political courage. Awards total $13,500 for winning essayists; the teacher who nominates the first-place winner will receive a $500 grant.

In other contest news, students can sign up now to compete in World Math Day on March 3.

Grading computer fails Churchill speech

Churchill’s speeches get low marks from a computerized grading system, British educators tell The Telegraph. “We shall fight on the beaches. We shall fight on the landing grounds. We shall fight in the fields and on the streets . . . ” Too repetitive.

His reference to the “might of the German army” lost him marks because the computer assumed that Churchill had intended to say “might have”, instead of using “might” as a noun.

Graham Herbert, deputy head of the Chartered Institute of Educational Assessors, said: “The computer was limited in its scope. It couldn’t cope with metaphor and didn’t understand the purpose of the speech.

Hemingway was rated “less than average” by the computer, which said he should include more detail. Anthony Burgess was judged “incomprehensible.”

Online marking of papers is being tested by exam boards and could be introduced within the next few years. It is already in use in America, where some children have learnt to write in a style which the computer appreciates, known as “schmoozing the computer”.

Via Core Knowledge Blog.

Essay-grading software lets teachers assign a lot more writing, concludes Teacher Magazine in a 2006 story. When students write more, they improve.

. . . Criterion and other essay-grading technologies have their limitations. They can’t judge the creativity of a writing style or the inventiveness of metaphors and symbolism. And I remain skeptical that artificial intelligence can effectively differentiate between a good essay and a truly excellent one.

(Teacher Aleeta) Johnson acknowledges that Criterion is not a good tool for very sophisticated writers. It wouldn’t appreciate the skill and creativity of a budding Shakespeare, for example.

But few students are budding Shakespeares.

Who shall test the test readers?

Poorly trained part-timers determine test scores that loom so large in education, writes Todd Farley in a New York Times op-ed. The author of Making the Grades, Farley was hired to score fourth-grade, state-wide reading comprehension tests when he was a graduate student in 1994.

One of the tests I scored had students read a passage about bicycle safety. They were then instructed to draw a poster that illustrated a rule that was indicated in the text. We would award one point for a poster that included a correct rule and zero for a drawing that did not.

The first poster I saw was a drawing of a young cyclist, a helmet tightly attached to his head, flying his bike over a canal filled with flaming oil, his two arms waving wildly in the air. I stared at the response for minutes. Was this a picture of a helmet-wearing child who understood the basic rules of bike safety? Or was it meant to portray a youngster killing himself on two wheels?

Some fellow scorers wanted to give full marks for understanding bicycle safety; others wanted to give a zero.

I realized then — an epiphany confirmed over a decade and a half of experience in the testing industry — that the score any student would earn mostly depended on which temporary employee viewed his response.

This is why multiple-choice tests can be more reliable than subjectively graded tests that rely on drawing (or writing) skills to measure reading comprehension.

I have a review copy of Farley’s book, which I plan to read very soon — along with the four other review books waiting for me. Maybe today! Anyhow, I vowed not to mention other people’s books without promoting my own book, Our School: The Inspiring Story of Two Teachers, One Big Idea and the Charter School That Beat the Odds.