Forget multiple-choice tests: The Watson computer technology could grade students’ writing quickly and cheaply, says Stanley S. Litow, president of the IBM International Foundation, in an interview with The Chronicle of Philanthropy.
. . . you could have long-answer questions, you could have the ability to grade lengthy paragraphs of information. If the testing system incorporates that, it will allow teachers to test to higher standards and children to learn at higher levels. And it will save lots of money in what is currently a very ineffective and inefficient testing and assessment system.
Computer grading of essays has been around for years, but critics doubt its accuracy.
The Educational Testing Service claims its E-Rater program accurately assessed the writing of freshmen at the New Jersey Institute of Technology, reports USA Today. ETS presented the “validity test” at a writing conference at George Mason University in February. E-Rater’s scores matched human graders’ assessments and the students’ SAT writing scores.
But a writing scholar at the Massachusetts Institute of Technology presented research questioning the ETS findings, and arguing that the testing service’s formula for automated essay grading favors verbosity over originality. Further, the critique suggested that ETS was able to get good results only because it tested short answer essays with limited time for students — and an ETS official admitted that the testing service has not conducted any validity studies on longer form, and longer timed, writing.
E-Rater has changed the behavior of NJIT students, said Andrew Klobucar, assistant professor of humanities. “First-year students are willing to revise essays multiple times when they are reviewed through the automated system, and in fact have come to embrace revision if it does not involve turning in papers to live instructors.”