As a 23-year-old “slacker” with a college degree, Kevin was hired by Pearson to score a sixth-grade math question on Illinois’ state exam. For six weeks, he scored the same question over and over again for eight hours a day.
Students are graded on a four-point scale on three dimensions: the actual answer they gave, the steps they took to get the answer, and the written explanation of why they took those steps. The first two dimensions were straightforward enough for the scorers, but the third one, the explanation, was a nightmare to score.
“The grading criteria for the explanation section changed three times,” Kevin writes.